datastore source pure

Package datastore provides support to store multiple collections of records.

Datastore Package

Package provides support to store multiple collections of records.

It supports the definition of multiple storages, where each one is a collection of records. Records can have any number of user defined fields which are added dynamically when values are set on a record. These fields can also be renamed or removed.

Storages have support for simple schemas that allow users to pre-define fields which can optionally have a default value also defined. Default values are assigned to new records on creation.

User defined schemas can optionally be strict, which means that records from a storage using the schema can only assign values to the pre-defined set of fields. In which case, assigning a value to an unknown field results in an error.

Package also supports the definition of custom record indexes. Indexes are used by storages to search and iterate records. The default index is the ID index but custom single and multi value indexes can be defined.

Warning

Using this package to store your realm data must be carefully considered. The fact that record fields are not strictly typed and can be renamed or removed could lead to issues if not careful when coding your realm(s). So it's recommended that you consider other alternatives first, like alternative patterns or solutions provided by the blockchain to deal with data, types and data migration for example.

Repository can be found at jeronimoalbi/gnome, as part of jeronimoalbi's Gno smart contracts monorepo.

Usage

 1package main
 2
 3import "gno.land/p/g17khqpukees4237dtn3astzapmp462vjhsz6st4/datastore"
 4
 5func main() {
 6	db := datastore.NewDatastore()
 7
 8	// Define a unique case insensitive index for user emails
 9	emailIdx := datastore.NewIndex("email", func(r datastore.Record) string {
10		return r.MustGet("email").(string)
11	}).Unique().CaseInsensitive()
12
13	// Create a new storage for user records
14	users := db.CreateStorage("users", datastore.WithIndex(emailIdx))
15
16	// Add a user with a single "email" field
17	user := users.NewRecord()
18	user.Set("email", "[email protected]")
19
20	// Save to assign the user ID and update indexes
21	user.Save()
22
23	// Find user by email using the custom index
24	user, _ = users.Get(emailIdx.Name(), "[email protected]")
25	println("Found by email:", user.MustGet("email"))
26
27	// Find user by ID
28	user, _ = users.GetByID(user.ID())
29	println("Found by ID:", user.ID())
30
31	// Delete the user from the storage and update indexes
32	users.Delete(user.ID())
33	println("Users:", users.Size())
34}
35
36// Output:
37// Found by email: [email protected]
38// Found by ID: 1
39// Users: 0

Querying

 1package main
 2
 3import "gno.land/p/g17khqpukees4237dtn3astzapmp462vjhsz6st4/datastore"
 4
 5func main() {
 6	db := datastore.NewDatastore()
 7
 8	// Define a multi value index to search posts by tag
 9	tagsIdx := datastore.NewMultiValueIndex("tags", func(r datastore.Record) []string {
10		return r.MustGet("tags").([]string)
11	})
12
13	posts := db.CreateStorage("posts", datastore.WithIndex(tagsIdx))
14	for _, title := range []string{"Post 1", "Post 2", "Post 3"} {
15		post := posts.NewRecord()
16		post.Set("title", title)
17		post.Set("tags", []string{"gno", title})
18		post.Save()
19	}
20
21	// Get two records starting from the second one
22	recordset, err := posts.Query(datastore.WithOffset(1), datastore.WithSize(2))
23	if err != nil {
24		panic(err)
25	}
26
27	recordset.Iterate(func(r datastore.Record) bool {
28		println("Post:", r.MustGet("title"))
29		return false // Continue iterating
30	})
31
32	// Get all the records that have the "gno" tag using the custom index
33	recordset = posts.MustQuery(datastore.UseIndex("tags", "gno"))
34	println("Tagged posts:", recordset.Size())
35}
36
37// Output:
38// Post: Post 2
39// Post: Post 3
40// Tagged posts: 3

Package datastore provides support to store multiple collections of records.

1const DefaultIndexOptions = collection.DefaultIndex | collection.SparseIndex

source

DefaultIndexOptions defines the default options for new indexes.

1var (
2	ErrEmptyQueryIndexName = errors.New("query index name is empty")
3	ErrInvalidQueryOffset  = errors.New("minimum allowed query offset is 0")
4	ErrInvalidQuerySize    = errors.New("minimum allowed query size is 1")
5)

source

1var ErrStorageExists = errors.New("a storage with the same name exists")

source

ErrStorageExists indicates that a storage exists with the same name.

1var ErrUndefinedField = errors.New("undefined field")

source

ErrUndefinedField indicates that a field in not defined in a record's schema.

1func NewDatastore() *Datastore

source

NewDatastore creates a new store.

1func NewIndex(name string, fn IndexFn) Index

source

NewIndex creates a new single value index.

Usage example:

Example

1// Index a User record by email
2idx := NewIndex("email", func(r Record) string {
3  return r.MustGet("email").(string)
4})

1func NewMultiValueIndex(name string, fn IndexMultiValueFn) Index

source

NewMultiValueIndex creates a new multi value index.

Usage example:

Example

1// Index a Post record by tag
2idx := NewMultiValueIndex("tag", func(r Record) []string {
3  return r.MustGet("tags").([]string)
4})

1func UseIndex(name, key string) QueryOption

source

UseIndex assigns the index that the query must use to get the records. Using an index requires a key value to locate the records within the index.

1func WithOffset(offset int) QueryOption

source

WithOffset assigns the offset or position of the first record that query must return. The minimum allowed offset is 0.

1func WithSize(size int) QueryOption

source

WithSize assigns the maximum number of records that query can return. The minimum allowed size is 1.

1func NewSchema(name string, options ...SchemaOption) *Schema

source

NewSchema creates a new schema.

1func Strict() SchemaOption

source

Strict configures the schema as a strict one. By default schemas should allow the creation of any user defined field, making them strict limits the allowed record fields to the ones pre-defined in the schema. Fields are pre-defined using `WithField`, `WithDefaultField` or by calling `Schema.AddField()`.

1func WithDefaultField(name string, value interface{}) SchemaOption

source

WithDefaultField assign a new field with a default value to the schema definition. Default value is assigned to newly created records asociated to to schema.

1func WithField(name string) SchemaOption

source

WithField assign a new field to the schema definition.

1func NewStorage(name string, options ...StorageOption) Storage

source

NewStorage creates a new records storage.

1func WithIndex(i Index) StorageOption

source

WithIndex assigns an index to the storage.

1func WithSchema(s *Schema) StorageOption

source

WithSchema assigns a schema to the storage.

1type Datastore struct {
2	storages *bptree.BPTree // string(name) -> *Storage
3}

source

Datastore is a store that can contain multiple named storages. A storage is a collection of records.

Example usage:

Example

1// Create an empty storage to store user records
2var db Datastore
3storage := db.CreateStorage("users")
4
5// Get a storage that has been created before
6storage = db.GetStorage("profiles")

1func (ds *Datastore) CreateStorage(name string, options ...StorageOption) *Storage

source

CreateStorage creates a new named storage within the data store.

1func (ds *Datastore) GetStorage(name string) *Storage

source

GetStorage returns a storage that has been created with a specific name. It returns nil when a storage with the specified name is not found.

1func (ds *Datastore) HasStorage(name string) bool

source

HasStorage checks if data store contains a storage with a specific name.

1type Index struct {
2	name    string
3	options collection.IndexOption
4	fn      interface{}
5}

source

Index defines a type for custom user defined storage indexes. Storages are by default indexed by the auto geneated record ID but can additionally be indexed by other custom record fields.

1func (idx Index) CaseInsensitive() Index

source

CaseInsensitive returns a copy of the index that indexes record values ignoring casing. Returned index contains previous options plus the case insensitivity one.

1func (idx Index) Func() interface{}

source

Func returns the function that storage collections apply to each record to get the value to use for indexing it.

1func (idx Index) Name() string

source

Name returns index's name.

1func (idx Index) Options() collection.IndexOption

source

Options returns current index options. These options define the index behavior regarding case sensitivity and uniquenes.

1func (idx Index) Unique() Index

source

Unique returns a copy of the index that indexes record values as unique values. Returned index contains previous options plus the unique one.

1type IndexFn func(Record) string

source

IndexFn defines a type for single value indexing functions. This type of function extracts a single string value from a record that is then used to index it.

1type IndexMultiValueFn func(Record) []string

source

IndexMultiValueFn defines a type for multi value indexing functions. This type of function extracts multiple string values from a record that are then used to index it.

1type Query struct {
2	offset    int
3	size      int
4	indexName string
5	indexKey  string
6}

source

Query contains arguments for querying a storage.

1func (q Query) IndexKey() string

source

IndexKey return the index key value to locate the records. An empty string is returned when all indexed records match the query.

1func (q Query) IndexName() string

source

IndexName returns the name of the storage index being used for the query.

1func (q Query) IsEmpty() bool

source

IsEmpty checks if the query is empty. Empty queries return no records.

1func (q Query) Offset() int

source

Offset returns the position of the first record to return. The minimum offset value is 0.

1func (q Query) Size() int

source

Size returns the maximum number of records a query returns.

1type QueryOption func(*Query) error

source

QueryOption configures queries.

 1type ReadOnlyRecord interface {
 2	// ID returns record's ID
 3	ID() uint64
 4
 5	// Key returns a string representation of the record's ID.
 6	// It's used to be able to search records within the ID index.
 7	Key() string
 8
 9	// Type returns the record's type.
10	Type() string
11
12	// Fields returns the list of the record's field names.
13	Fields() []string
14
15	// IsEmpty checks if the record has no values.
16	IsEmpty() bool
17
18	// HasField checks if the record has a specific field.
19	HasField(name string) bool
20
21	// Get returns the value of a record's field.
22	Get(field string) (value interface{}, found bool)
23
24	// MustGet returns the value of a record's field or panics when the field is not found.
25	MustGet(field string) interface{}
26}

source

ReadOnlyRecord defines an interface for read-only records.

 1type Record interface {
 2	ReadOnlyRecord
 3
 4	// Set assings a value to a record field.
 5	// If the field doesn't exist it's created if the underlying schema allows it.
 6	// Storage schema can optionally be strict in which case no new fields other than
 7	// the ones that were previously defined are allowed.
 8	Set(field string, value interface{}) error
 9
10	// Save assigns an ID to newly created records and update storage indexes.
11	Save() bool
12}

source

Record stores values for one or more fields.

1type RecordIterFn func(Record) (stop bool)

source

RecordIterFn defines a type for record iteration functions.

 1type Recordset interface {
 2	// Iterate iterates records in order.
 3	Iterate(fn RecordIterFn) (stopped bool)
 4
 5	// ReverseIterate iterates records in reverse order.
 6	ReverseIterate(fn RecordIterFn) (stopped bool)
 7
 8	// Size returns the number of records in the recordset.
 9	Size() int
10}

source

Recordset defines an interface that allows iterating multiple records.

1type Schema struct {
2	name     string
3	strict   bool
4	fields   list.List      // int(field index) -> string(field name)
5	defaults *bptree.BPTree // string(field index) -> interface{}
6}

source

Schema contains information about fields and default field values. It also offers the possibility to configure it as static to indicate that only configured fields should be allowed.

1func (s *Schema) AddField(name string, defaultValue interface{}) (index int, added bool)

source

AddField adds a new field to the schema. A default field value can be specified, otherwise `defaultValue` must be nil.

1func (s *Schema) Fields() []string

source

Fields returns the list field names that are defined in the schema.

1func (s *Schema) GetDefault(name string) (value interface{}, found bool)

source

GetDefault returns the default value for a field.

1func (s *Schema) GetDefaultByIndex(index int) (value interface{}, found bool)

source

GetDefaultByIndex returns the default value for a field by it's index.

1func (s *Schema) GetFieldIndex(name string) int

source

GetFieldIndex returns the index number of a schema field.

Field index indicates the order the field has within the schema. When defined fields are added they get an index starting from field index 0.

Fields are internally referenced by index number instead of the name to be able to rename fields easily.

1func (s *Schema) GetFieldName(index int) (name string, found bool)

source

GetFieldName returns the name of a field for a specific field index.

1func (s *Schema) HasField(name string) bool

source

HasField check is a field has been defined in the schema.

1func (s *Schema) IsStrict() bool

source

IsStrict check if the schema is configured as a strict one.

1func (s *Schema) Name() string

source

Name returns schema's name.

1func (s *Schema) RenameField(name, newName string) (renamed bool)

source

RenameField renames a field.

1func (s *Schema) Size() int

source

Size returns the number of fields the schema has.

1type SchemaOption func(*Schema)

source

StorageOption configures schemas.

1type Storage struct {
2	name       string
3	collection *collection.Collection
4	schema     *Schema
5}

source

Storage stores a collection of records.

By default it searches records by record ID but it allows using custom user defined indexes for other record fields.

When a storage is created it defines a default schema that keeps track of record fields. Storage can be optionally created with a user defined schema in cases where the number of fields has to be pre-defined or when new records must have one or more fields initialized to default values.

1func (s Storage) Collection() *collection.Collection

source

Collection returns the undelying collection used by the storage to store all records.

1func (s Storage) Delete(id uint64) bool

source

Delete deletes a record from the storage.

1func (s Storage) Get(indexName, indexKey string) (_ Record, found bool)

source

Get returns the first record found for a key within a storage index.

This is a convenience method to get a single record. A multi index will always return the first record value for the specified key in this case. To get multiple records create a query using a custom index and key value or use the underlying storage collection.

1func (s Storage) GetByID(id uint64) (_ Record, found bool)

source

GetByID returns a record whose ID matches the specified ID.

1func (s Storage) MustQuery(options ...QueryOption) Recordset

source

MustQuery returns a recordset that matches the query parameters or panics on error. By default query selects records using the ID index.

Example usage:

Example

1// Get 50 records starting from the one at position 100
2var records []Record
3storage.MustQuery(
4	WithOffset(100),
5	WithSize(50),
6).Iterate(func (r Record) bool {
7	records = append(records, r)
8	return false
9})

1func (s Storage) Name() string

source

Name returns storage's name.

1func (s Storage) NewRecord() Record

source

NewRecord creates a new storage record.

If a custom schema with default field values is assigned to storage it's used to assign initial default values when new records are created.

Creating a new record doesn't assign an ID to it, a new ID is generated and assigned to the record when it's saved for the first time.

1func (s Storage) Query(options ...QueryOption) (Recordset, error)

source

Query returns a recordset that matches the query parameters. By default query selects records using the ID index.

Example usage:

Example

 1// Get 50 records starting from the one at position 100
 2rs, _ := storage.Query(
 3	WithOffset(100),
 4	WithSize(50),
 5)
 6
 7// Iterate records to create a new slice
 8var records []Record
 9rs.Iterate(func (r Record) bool {
10	records = append(records, r)
11	return false
12})

1func (s Storage) Schema() *Schema

source

Schema returns the schema being used to track record fields.

1func (s Storage) Size() int

source

Size returns the number of records that the storage have.

1type StorageOption func(*Storage)

source

StorageOption configures storages.

Readme View source

Datastore Package

Usage

Querying

Overview

Constants 1

Variables 3

Functions 13

Types 14

Methods on Datastore

Methods on Index

Methods on Query

Methods on Schema

Methods on Storage

Imports 7

Source Files 11