This morning the words "computational storage mesh" popped into my head as yet another potential descriptor for the elusive, vaporware idea of "dataspace" that I've been pursuing for years.

Turns out that "computational storage" (without the mesh) is an actual buzzword trending right now. Huh.

nextplatform.com/2020/02/25/co

What I want by a "computational storage mesh" is:

1. To start with, a big block-structured arbitrary storage space that you can shove stuff into and get stuff out of. Probably hash-addressed, but with arbitrarily other indexes into it.

2. Some blocks of data might be computed. How that works, what language, what semantics, that's the problem. Probably some form of lazy functional language.

3. Data and computational must remain *physically local* wherever possible, so not the "cloud" model.

The simplest top-level semantics for getting computation out of a block-storage API is probably something like:

"instead of literally retrieving block with identifier XYZ, instead evaluate block with identifier XYZ, and give me the block representing the literal contents of that evaluation".

But if "evaluate" implies "recursive evaluation of subexpressions", then even this could turn into infinite recursive storage and time-consuming logic bombs, so limits probably need to be part of the API.

Follow

@natecull what about rocksdb? Used by Facebook but seems benign. It backs my XTDB and is fast, key-value pair, totally local, built to be scalable.

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.