Data Sources

This module provides base classes for data sources. Data sources provide data to calculations. All data used comes from a data source. The requirements for data sources are as follows:

1. Data sources must be sub-classed to DataSource.

2. They must know where to get their data, either from a file or from other

   data sources.

3. They need a data reader that knows how to extract the data from the file,

   or combine data in calculations to produce new data.

4. They require a parameter map that states exactly where the data is and what

   its units are, what the data will be called in calculations and any other meta-data the registry requires.

Data Parameter

class simkit.core.data_sources.DataParameter(*args, **kwargs)

Field for data parameters.

Parameters

• units – units of this data parameter

• uncertainty – uncertainty

• isconstant (bool) – true if doesn’t vary in dynamic simulations

• timeseries – index of dynamic simulation inputs

Data Regsitry

class simkit.core.data_sources.DataRegistry

A registry for data sources. The meta names are: uncertainty, variance, isconstant, timeseries and data_source

meta_names = ['uncertainty', 'variance', 'isconstant', 'timeseries', 'data_source']

meta names

register(newdata, *args, **kwargs)

Register data in registry. Meta for each data is specified by positional or keyword arguments after the new data and consists of the following:

• uncertainty - Map of uncertainties in percent corresponding to new keys. The uncertainty keys must be a subset of the new data keys.

• variance - Square of the uncertainty (no units).

• isconstant: Map corresponding to new keys whose values are``True`` if constant or False if periodic. These keys must be a subset of the new data keys.

• timeseries: Name of corresponding time series data, None if no time series. _EG_: DNI data timeseries attribute might be set to a date/time data that it corresponds to. More than one data can have the same timeseries data.

• data_source: the DataSource superclass that was used to acquire this data. This can be used to group data from a specific source together.

Parameters

newdata (mapping) – New data to add to registry. When registering new data, keys are not allowed to override existing keys in the data registry.

Raises

UncertaintyPercentUnitsError

Data Source Base

class simkit.core.data_sources.DataSourceBase

Base data source meta class.

Data Source

class simkit.core.data_sources.DataSource(*args, **kwargs)

Required interface for all SimKit data sources such as PVSim results, TMY3 data and calculation input files.

Each data source must specify a data_reader which must subclass DataReader and that can read this data source. The default is JSONReader.

Each data source must also specify a data_file and data_path that contains the parameters required to import data from the data source using the data reader. Each data reader had different parameters to specify how it reads the data source, so consult the API.

This is the required interface for all source files containing data used in SimKit.

args = None

positional arguments

data = None

data loaded from reader

data_source = None

name of DataSource

edit(edits, data_reg)

Edit data in Data_Source. Sets issaved to False.

filename = None

filename of file containing data

isconstant = None

True if data is constant for all dynamic calculations

kwargs = None

keyword arguments

saveas_json(save_name)

Save data, param_file, original data_reader and UTC modification time as keys in JSON file. If data is edited then it should be saved using this method. Non-JSON data files are also saved using this method.

Parameters

save_name (str) – Name to save JSON file as, “.json” is appended.

timeseries = None

name of corresponding time series data, None if no time series

uncertainty = None

data uncertainty in percent

variance = None

variance

Data Readers

This module provides the base classes for data readers, such as XLRD and numpy.loadtxt(), which are used to read in data sources.

DataReader

class simkit.core.data_readers.DataReader(parameters, meta=None)

Required interface for all SimKit data readers.

Parameters

parameters (dict) – parameters to be read

apply_units_to_cache(data)

Apply units to cached data. This method must be implemented by each data reader.

Parameters

data – cached data

Returns

data with units applied

Return type

Quantity

Raises

NotImplementedError

is_file_reader = True

True if reader accepts filename argument

load_data(*args, **kwargs)

Load data from source using reader. This method must be implemented by each data reader.

Parameters

• args – positional arguments

• kwargs – keyword arguments

Returns

data read by DataReader

Return type

dict

Raises

NotImplementedError

meta = None

meta if any

parameters = None

parameters to be read by reader

JSONReader

class simkit.core.data_readers.JSONReader(parameters, meta=None)

Read data from a JSON file.

Parameters

• parameters (dict) – parameters to read

• data_reader – original DataReader if data cached as JSON

This the default data reader if not specified in the data source. The format of the data is similar to the dictionary used to create the data registry, except without units.

For example:

{
    "data": {
        "DNI": [834, 523, 334, 34, 0, 0],
        "zenith": [21, 28, 45, 79, 90, 90]
    },
    "param_file": "path/to/corresponding/param_file.json",
    "data_source": "MyDataSource"
}

Parameters can be specified in a JSON file.

{
    "DNI": {
        "description": "direct normal insolation",
        "units": "W/m*^2",
        "isconstant": false
    },
    "zenith": {
        "description": "solar zenith",
        "units": "degrees",
        "isconstant": false
    }
}

Parameters can also be specified in the data source as class attributes.

class MyDataSrc(DataSource):
    data_reader = JSONReader
    DNI = {
        "description": "direct normal insolation",
        "units": "W/m*^2",
        "isconstant": false
    }
    zenith = {
        "description": "solar zenith",
        "units": "degrees",
        "isconstant": false
    }

apply_units_to_cache(data)

Apply units to data read using JSONReader.

Parameters

data – cached data

Returns

data with units applied

Return type

Quantity

load_data(filename, *args, **kwargs)

Load JSON data.

Parameters

filename (str) – name of JSON file with data

Returns

data

Return type

dict

orig_data_reader = None

original data reader [None]