sift_py.data_import

  1"""
  2This module contains services to facilitate importing data.
  3It also provides utilities to easily query the import status.
  4
  5The fundamental components of this module are the following:
  6- `sift_py.data_import.config.CsvConfig`
  7- `sift_py.data_import.csv.CsvUploadService`
  8- `sift_py.data_import.status.DataImportService`
  9
 10
 11## Simple CSV Upload
 12
 13A simple CSV upload without needing to craft a custom CSV config can be done like so:
 14```python
 15from sift_py.data_import.csv import CsvUploadService
 16from sift_py.data_import.status import DataImportService
 17from sift_py.rest import SiftRestConfig
 18
 19rest_config: SiftRestConfig = {
 20    "uri": sift_uri,
 21    "apikey": apikey,
 22}
 23
 24asset_name = "Your Asset Name"
 25csv_upload_service = CsvUploadService(rest_config)
 26import_service: DataImportService  = csv_upload_service.simple_upload(asset_name, "sample_data.csv")
 27
 28# Blocks until the import is completed.
 29import_service.wait_until_complete()
 30```
 31
 32This example assumes several things about how the data is formatted. For example, that first column
 33contains datetime formatted time stamps. See docstring for `simple_upload` to see what can be overridden.
 34
 35## TDMS Upload
 36
 37Specify `sift-stack-py[tdms]` in your dependencies to use the TDMS upload service.
 38TDMS files can be uploaded like so:
 39```python
 40from sift_py.data_import.csv import TdmsUploadService
 41from sift_py.data_import.status import DataImportService
 42from sift_py.rest import SiftRestConfig
 43
 44rest_config: SiftRestConfig = {
 45    "uri": sift_uri,
 46    "apikey": apikey,
 47}
 48
 49asset_name = "Your Asset Name"
 50csv_upload_service = CsvUploadService(rest_config)
 51import_service: DataImportService  = csv_upload_service.simple_upload(asset_name, "sample_data.tdms")
 52
 53# Blocks until the import is completed.
 54import_service.wait_until_complete()
 55```
 56If you want to upload TDMS groups prefixes to channel names set `prefix_channel_with_group` to True:
 57```python
 58csv_upload_service.simple_upload(asset_name, "sample_data.tdms", prefix_channel_with_group=True)
 59```
 60
 61Some times there are TDMS channels without valid data or timing information, you can skip these channels by
 62setting `ignore_errors` to True:
 63```python
 64csv_upload_service.simple_upload(asset_name, "sample_data.tdms", ignore_errors=True)
 65```
 66The channels being skipped will be printed out to stdout.
 67
 68## CSV Upload with custom CSV config
 69
 70If your data is formatted a specific way you can create a CsvConfig that will be used to properly
 71parse your data:
 72```python
 73from sift_py.data_import.csv import CsvUploadService
 74from sift_py.data_import.status import DataImportService
 75from sift_py.rest import SiftRestConfig
 76from sift_py.data_import.config import CsvConfig
 77
 78rest_config: SiftRestConfig = {
 79    "uri": sift_uri,
 80    "apikey": apikey,
 81}
 82
 83csv_upload_service = CsvUploadService(rest_config)
 84
 85# Create CSV config.
 86input_csv = "sample_data.csv"
 87
 88# Parse CSV to get channel names.
 89data_config = {}
 90with open(input_csv, "r") as f:
 91    reader = csv.DictReader(f)
 92    headers = next(reader)
 93    for i, channel in enumerate(headers):
 94        if channel == "timestamp":
 95            continue
 96        data_config[i + 1] = {
 97            "name": channel,
 98            # This example assumes all channels are doubles.
 99            # Can also use `ChannelDoubleType.DOUBLE` or `double`
100            "data_type": "CHANNEL_DATA_TYPE_DOUBLE",
101            "description": f"Example channel {channel}",
102        }
103
104csv_config = CsvConfig(
105    {
106        "asset_name": asset_name,
107        "first_data_row": 2,
108        "time_column": {
109            "format": "TIME_FORMAT_ABSOLUTE_DATETIME",
110            # Can also use `TimeFormatType.ABSOLUTE_DATETIME`
111            "column_number": 1,
112        },
113        "data_columns": data_config,
114    }
115)
116
117import_service: DataImportService = csv_upload_service.upload(input_csv, csv_config)
118import_service.wait_until_complete()
119```
120
121In this example the CSV can be created programmatically. You can also import use a json file directly:
122```python
123import json
124from sift_py.data_import.config import CsvConfig
125
126with open("config.json") as f:
127    csv_config = CsvConfig(json.load(f))
128```
129"""