Superutils

Appearance

Class: DataStorage<Key, Value, CacheDisabled> ​

Defined in: packages/rx/src/data-storage/DataStorage.ts:58

Type Parameters ​

Key ​

Key extends StorageKey

Value ​

Value extends StorageValue

CacheDisabled ​

CacheDisabled extends boolean = false

Implements ​

Constructors ​

Constructor ​

new DataStorage<Key, Value, CacheDisabled>(name?, options?): DataStorage<Key, Value, CacheDisabled>

Defined in: packages/rx/src/data-storage/DataStorage.ts:179

A wrapper for reading and writing to LocalStorage (browser) or JSON files (NodeJS), providing a Map-like interface with advanced features like search, filtering, and sorting.

Notes: ​

  • Performance: DataStorage is optimized for small to medium datasets.
    • For datasets > 1MB, consider increasing the delay option to reduce write frequency.
    • It is NOT recommended for datasets larger than 3MB due to synchronous serialization costs.
  • RxJS Integration: Built on RxJS for reactive data handling, though no prior RxJS knowledge is required.
  • Storage Behavior:
    • If name is omitted, the instance operates in-memory only and data is not persisted to storage.
    • If cacheDisabled is true, data is not kept in memory; every read/write operation accesses the underlying storage directly.

Parameters ​

name? ​

string | null

options? ​

StorageOptions<Key, Value, CacheDisabled>

Returns ​

DataStorage<Key, Value, CacheDisabled>

Examples ​

Browser Usage ​

javascript
import { DataStorage } from '@superutils/rx'
import fetch from '@superutils/fetch'

const storage = new DataStorage('products')
const { products } = await fetch('[DUMMYJSON-DOT-COM]/products')
// save all items to storage
storage.setAll(
  new Map(products.map(p => [p.id, p])), // convert to Map
)

// print product with id `1`
console.log(storage.get(1))

// search for items
const searchResult = storage.search({
  query: { availabilityStatus: 'low' }
})
console.log(searchResult)

NodeJS Usage ​

javascript
import { DataStorage } from '@superutils/rx'
import fetch from '@superutils/fetch'
import { LocalStorage } from 'node-localstorage'

// Add localStorage alternative for NodeJS that reads and writes to JSON files.
// This is not necessary for browsers.
globalThis.localStorage = new LocalStorage('./data', 1e7)

const storage = new DataStorage('products')
const { products } = await fetch('[DUMMYJSON-DOT-COM]/products')
// save all items to storage
storage.setAll(
  new Map(products.map(p => [p.id, p])), // convert to Map
)

// print product with id `1`
console.log(storage.get(1))

// search for items
const searchResult = storage.search({
  query: { availabilityStatus: 'low' }
})
console.log(searchResult)

Advanced: onChange and RxJS subject ​

Internally, DataStorage uses RxJS subject which is exposed as subject property. You can use this to subscribe to changes and do additional operations such as logging or sanitization etc.

Alternatively, you can also set the onChange callback which is triggered whenever the subject changes and does not require maintaining a subscription or knowledge of RxJS subject.

javascript
import { DataStorage } from '@superutils/rx'

const storage = new DataStorage('my-data')
const sub = storage.subject.subscribe(data => {
  // Write to the database whenever data changes
  console.log('Saving to database...', data)
})
// unsubscribe from subject
setTimeout(()=> sub.unsbuscribe(), 1000)

// add an entry to storage
storage.set('bob', { age: 99, id: 'bob', name: 'Bob' })

Properties ​

cacheDisabled ​

readonly cacheDisabled: CacheDisabled

Defined in: packages/rx/src/data-storage/DataStorage.ts:63

Disable in-memory cache and only directly read/write from storage (local storage or JSON fle)

Implementation of ​

IDataStorage.cacheDisabled

delay ​

readonly delay: number

Defined in: packages/rx/src/data-storage/DataStorage.ts:65

Debounce/throttle delay duration in milliseconds for writing to storage when caching is enabled.

Increasing this value can improve performance when dealing with large datasets or frequent updates by reducing the number of write operations.

Default: 300

Implementation of ​

IDataStorage.delay

delayOptions? ​

readonly optional delayOptions: DelayOptions

Defined in: packages/rx/src/data-storage/DataStorage.ts:68

Debounce and throttle related options

Implementation of ​

IDataStorage.delayOptions

filter ​

readonly filter: StorageFilter<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:248

Filter items by predicate

Implementation of ​

IDataStorage.filter

find ​

readonly find: StorageFind<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:242

Find an item by predicate or search criteria

Implementation of ​

IDataStorage.find

initialized ​

readonly initialized: boolean = false

Defined in: packages/rx/src/data-storage/DataStorage.ts:70

Indicates wherether storage has been initialized (init() function invoked).

Implementation of ​

IDataStorage.initialized

map ​

readonly map: StorageMap<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:351

Map each item on the data to an Array

Implementation of ​

IDataStorage.map

name ​

readonly name: string

Defined in: packages/rx/src/data-storage/DataStorage.ts:72

Storage name. Filename (NodeJS) or property name (browser LocalStorage). If empty string or undefined, data will not be saved to storage and will only work in-memory.

Default: ''

Implementation of ​

IDataStorage.name

onChange? ​

optional onChange: StorageOnChangeFn<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:356

Callback to be invoked whenever value change is triggered.

If onChange invocation fails, it will be ignored gracefully.

Implementation of ​

IDataStorage.onChange

onError? ​

optional onError: StorageOnErrorFn

Defined in: packages/rx/src/data-storage/DataStorage.ts:358

Callback to be invoked whenever read/write operation fails.

If onError invocation failure will be ignored gracefully.

Implementation of ​

IDataStorage.onError

parse? ​

readonly optional parse: StorageParseFn<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:360

A callback to customize the deserialization of data read from storage.

This can be used to override the default JSON.parse behavior and serves as the counterpart to stringify. If this function is not provided, throws an error, or returns undefined, the default JSON.parse will be used as a fallback.

Implementation of ​

IDataStorage.parse

readonly search: StorageSearch<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:383

Search items

Implementation of ​

IDataStorage.search

sort ​

readonly sort: StorageSort<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:401

Sort items in the storage.

Param ​

Criteria to sort by. Accepts one of the following:

  • function: A comparator function to sort the data.
  • string: A property name of the value object to sort by.
  • true: Sorts the map by its keys.

Param ​

(optional) Sorting options.

Param ​

(optional) Whether to save the sorted data back to storage (localStorage/file).

Returns ​

The sorted Map.

Implementation of ​

IDataStorage.sort

spaces? ​

optional spaces: number

Defined in: packages/rx/src/data-storage/DataStorage.ts:78

Number of spaces to use when stringifying. Default: undefined

Implementation of ​

IDataStorage.spaces

storage? ​

readonly optional storage: StorageCompact | null

Defined in: packages/rx/src/data-storage/DataStorage.ts:80

LocalStorage or equivalent storage instance to be used as the underlying storage and to read & write from.

Notes:

  • Ignored when name is falsy (in-memory only mode)
  • For NodeJS or equivalent, an instance of LocalStorage from "node-localstoarge" NPM module can be used.
  • If undefined, will not attempt to use globalThis.localStorage, if available
  • If null, will not attempt to use globalThis.localStorage

Default:

  • browser: localStorage
  • node: undefined (in-memory mode)

Implementation of ​

IDataStorage.storage

stringify? ​

readonly optional stringify: StorageStringifyFn<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:415

Callback function to customize the serialization of data to be stored to storage.

Useful when data needs to be sanitised before storing and/or remove circlar references.

Example ​

javascript
import fetch from '@superutils/fetch'
import { DataStorage } from '@superutils/rx'
import { LocalStorage } from 'node-localstorage'

// Create a localStorage alternative for NodeJS that reads and writes to JSON files.
// This is not necessary for browsers
globalThis.localStorage = new LocalStorage('./data', 1e7)

const storage = new DataStorage('products.json')
storage.stringify = data => Array
	.from(data)
 .map(([key, product]) => [
   key,
   { id: product.id, title: product.title } // only store what's needed
 ])

const { products } = await fetch('[DUMMYJSON-DOT-COM]/products)
const productsMap = result.products.map(p => [p.id, p])
storage.setAll(productsMap, true)
console.log(storage.getAll())

Implementation of ​

IDataStorage.stringify

subject ​

readonly subject: CacheDisabled extends true ? Subject<Map<Key, Value>> : BehaviorSubject<Map<Key, Value>>

Defined in: packages/rx/src/data-storage/DataStorage.ts:82

The underlying RxJS Subject that serves as the primary reactive interface for observing data modifications.

Its implementation type is determined by the caching strategy:

  • BehaviorSubject: Used when caching is enabled. It maintains the current state and emits it immediately to new subscribers.
  • Subject: Used when caching is disabled. It acts as a pure event pipe, emitting updates only at the moment they occur without retaining an in-memory copy.

Implementation of ​

IDataStorage.subject

toJSON ​

readonly toJSON: StorageToJSON<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:419

Convert list of items (Map) to JSON string of 2D Array

Implementation of ​

IDataStorage.toJSON

Accessors ​

size ​

Get Signature ​

get size(): number

Defined in: packages/rx/src/data-storage/DataStorage.ts:74

Get the number of items

Returns ​

number

Get the number of items

Implementation of ​

IDataStorage.size

Methods ​

clear() ​

readonly clear(): DataStorage<Key, Value, CacheDisabled>

Defined in: packages/rx/src/data-storage/DataStorage.ts:227

Clear all items

Returns ​

DataStorage<Key, Value, CacheDisabled>

Implementation of ​

IDataStorage.clear

delete() ​

readonly delete(keys): DataStorage<Key, Value, CacheDisabled>

Defined in: packages/rx/src/data-storage/DataStorage.ts:232

Delete one or more items by their respective keys

Parameters ​

keys ​

Key | Key[]

Returns ​

DataStorage<Key, Value, CacheDisabled>

Implementation of ​

IDataStorage.delete

get() ​

readonly get(key): Value | undefined

Defined in: packages/rx/src/data-storage/DataStorage.ts:278

Get item by key

Parameters ​

key ​

Key

Returns ​

Value | undefined

Implementation of ​

IDataStorage.get

getAll() ​

readonly getAll(forceRead): Map<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:280

Get all items

Parameters ​

forceRead ​

boolean = false

Returns ​

Map<Key, Value>

Implementation of ​

IDataStorage.getAll

has() ​

readonly has(key): boolean

Defined in: packages/rx/src/data-storage/DataStorage.ts:295

Check if key exists

Parameters ​

key ​

Key

Returns ​

boolean

Implementation of ​

IDataStorage.has

init() ​

readonly init(initialValue?): boolean

Defined in: packages/rx/src/data-storage/DataStorage.ts:297

Parameters ​

initialValue? ​

Map<Key, Value>

An optional map to initialize the storage with if it's currently empty.

Returns ​

boolean

true if initialization was successful, or false if the storage was already initialized.

Implementation of ​

IDataStorage.init

keys() ​

readonly keys(): Key[]

Defined in: packages/rx/src/data-storage/DataStorage.ts:349

Get all keys

Returns ​

Key[]

Implementation of ​

IDataStorage.keys

read() ​

readonly read(): Map<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:362

Read directly from the localStorage (browser) or file (NodeJS) without triggering the this.subject.

Returns ​

Map<Key, Value>

Implementation of ​

IDataStorage.read

set() ​

readonly set(key, value): DataStorage<Key, Value, CacheDisabled>

Defined in: packages/rx/src/data-storage/DataStorage.ts:386

Set item by key

Parameters ​

key ​

Key

value ​

Value

Returns ​

DataStorage<Key, Value, CacheDisabled>

Implementation of ​

IDataStorage.set

setAll() ​

readonly setAll(data, replace): DataStorage<Key, Value, CacheDisabled>

Defined in: packages/rx/src/data-storage/DataStorage.ts:391

Set multiple entries at once and/or replace the storage entries

Parameters ​

data ​

Map<Key, Value> = ...

replace ​

boolean = false

Returns ​

DataStorage<Key, Value, CacheDisabled>

Implementation of ​

IDataStorage.setAll

toArray() ​

readonly toArray(): [Key, Value][]

Defined in: packages/rx/src/data-storage/DataStorage.ts:417

Convert list of items (Map) to 2D Array

Returns ​

[Key, Value][]

Implementation of ​

IDataStorage.toArray

toObject() ​

readonly toObject(): Record<Key, Value>

Defined in: packages/rx/src/data-storage/DataStorage.ts:440

Convert list of items into an object

Returns ​

Record<Key, Value>

Implementation of ​

IDataStorage.toObject

toString() ​

readonly toString(data?): string

Defined in: packages/rx/src/data-storage/DataStorage.ts:447

Convert list of items (Map) to JSON string of 2D Array

Parameters ​

data? ​

Map<Key, Value>

Returns ​

string

Implementation of ​

IDataStorage.toString

unsubscribe() ​

readonly unsubscribe(): void

Defined in: packages/rx/src/data-storage/DataStorage.ts:455

Unsubscribe from all internal subscriptions.

This will result in:

  • Automatic writing to storage being disabled (manual writes via instance.write() will still work).
  • The onChange callback no longer being triggered.
  • The instance stopping listening to force update cache triggers.

Returns ​

void

Implementation of ​

IDataStorage.unsubscribe

values() ​

readonly values(): Value[]

Defined in: packages/rx/src/data-storage/DataStorage.ts:457

Get all values as an array

Returns ​

Value[]

Implementation of ​

IDataStorage.values

write() ​

readonly write(data?): boolean

Defined in: packages/rx/src/data-storage/DataStorage.ts:459

Parameters ​

data? ​

Map<Key, Value>

(optional) Data to write.

  • If provided, it overwrites the storage.
  • If not provided, the current in-memory data is used (if cache is enabled).

Returns ​

boolean

true if the write was successful, false otherwise.

Implementation of ​

IDataStorage.write

forceUpdateCache() ​

static forceUpdateCache(name): void

Defined in: packages/rx/src/data-storage/DataStorage.ts:274

Trigger forced update of cached data from storage.

Parameters ​

name ​

determines which storage instances to be updated.

  • name (string | string[]): update all instances with a specific name(s)
  • global (true): update all instances globally

string | true | string[]

Returns ​

void

Example ​

javascript
import { DataStorage } from '@superutils/rx'

// Update all DataStorage instances with specific name(s)
const name = 'products'
DataStorage.forceUpdateCache([name])

// Update every single instance of DataStorage that uses storage (has a "name")
DataStorage.forceUpdateCache(true)