Examples

The examples contained on this page are designed to show you how to build a record for training and inference in Zeff Cloud API.

The above links will have details on how to build a record from a relational database (rdbms), a CSV file (csv), a YAML file (yaml), or to create a new clean project from scratch.

This readme applies to all examples and will explain the overall process for building a record and how a record relates to other objects.

Process

@startuml
ZeffClient -> RecordConfigGenerator: next()
RecordConfigGenerator --> ZeffClient: config_string
ZeffClient -> RecordBuilder: config_string
RecordBuilder -> RecordBuilder: create record with name
RecordBuilder -> RecordBuilder: create and add structured data items
RecordBuilder -> RecordBuilder: create and add unstructured data items
RecordBuilder --> ZeffClient: record
@enduml

ZeffClient

This is the application that will coordinate building, validation, and uploading of records to the Zeff Cloud API. This is meant to reduce the amount of infrastructure you need to build in order to use Zeff Cloud.

Record Config Generator

This is a generator object you will build that will generate configuration strings. These strings will be used by ZeffClient as a parameter to a record builder in order to create a single record. This is not necessary when running a record builder directly, but is needed by ZeffClient.

Having this separate from the record builder allows ZeffClient to control record building so multiple records may be built in parallel to allow improved throughput to Zeff Cloud.

The configuration strings may contain anything you choose, but must be acceptable to the associated record builder when it is called.

There are generators in zeff.recordgenerator package that may be used for common types of configuration strings such as URLs. For generators that work from RDBMS, YAML files, CSV files, etc. a custom generator must be created.

Record Builder

This is a class that you will create that ZeffClient will create a record builder object. These objects must be callable with a flag that indicates if a dataset or model record is being created, a configuration string parameter that has been generated from a record config generator, and will return a single record object.

If the builder is created with zeff init then it will contain code that will allow it to be executed as a stand alone script that will take a string generated by the config generator and will print out a table that shows what is in the record.

Record

A record object is built from classes in zeff.record package. An object of this type is required to be returned from your record builder.

A record object is a composition of structured and unstructured data.

@startuml

namespace ZeffClient.genericrecord {
	class Record {
		name: String
		void validate()
		void submit()
	}
	Record "1" *-- "0..*" StructuredData
	Record "1" *-- "0..*" UnstructuredData

	class StructuredData {
		name: String
		value: String, Int, Float, Bool
		data_type: [continuous, discrete]
		target: [yes, no, ignore]
	}

	class UnstructuredData{
		data: URI
		file_type: [image, video, audio, document, text, meta]
		group_by: String
		embeddata: Bool
	}
}

@enduml

Validator

A validator is a callable object that will be given a record to determine if it will be accepted by the ZeffCloud system. If additional validation is desired then a custom validator may be created.