Struct libriakv::RiaKV

pub struct RiaKV<F> where
    F: Read + Write + Seek,  {
    f: F,
    pub index: HashMap<ByteString, u64>,
}

Generic representation of a key value store in libriakv The underlying storage can be any type with the Read + Write + Seek trait bounds. The index attribute is used to maintain a mapping from keys to the position in the underlying storage where their corresponding entries are stored.

Fields

f: F

underlying storage

index: HashMap<ByteString, u64>

index - storing a mapping from keys to the position where the key value entry is stored

Implementations

impl RiaKV<File>

pub fn open_from_file_at_path(path: &Path) -> Result<Self>

Creates a new RiaKV instance from a file stored at the given path as th backing store.

Example

use libriakv::RiaKV;

let storage_path = std::path::Path::new("/path/to/some/file.db");

match RiaKV::open_from_file_at_path(storage_path) {
    Ok(opened_store) => {}, // use the opened store
    _ => {} // handle failure
};

impl RiaKV<Cursor<Vec<u8>>>

pub fn open_from_in_memory_buffer(capacity: usize) -> Self

Creates a new RiakV instance from an in memory buffer Vec<u8> with the given capacity.

Example

use libriakv::RiaKV;

let mut store = RiaKV::open_from_in_memory_buffer(5000);

impl<F> RiaKV<F> where
    F: Read + Write + Seek, 

pub fn process_record<R: Read>(f: &mut R) -> Result<KeyValuePair>

Processes a record from the current position in the underlying storage file. Every record (key value pair) is stored with the following layout:

┌────────────────┬────────────┬──────────────┬────────────────┐
│ crc32 checksum │ key length │ value length │ KeyValuePair{} │
└────────────────┴────────────┴──────────────┴────────────────┘

Reading a record from the underlying storage occurs in the following steps:

• Read the checksum, key length and value length as 32 bit integers with little endian format
• Read the next (key length + value length) bytes into a bytestring
• Verify that the crc32 checksum of the data Bytestring read matches with the crc32 checksum read
• Split off the bytestring at key length from the start to obtain the key and the value
• Return KeyValuePair { key, value }

Example

use std::io;
use libriakv::RiaKV;

let mut cursor = io::Cursor::new(vec![0; 5000]);

// .. enter some data into the cursor

let maybe_kv = RiaKV::<io::Cursor<Vec<u8>>>::process_record(&mut cursor);

pub fn seek_to_end(&mut self) -> Result<u64>

Seeks to the end of the underlying storage file. Any subsequent read should end in EOF.

pub fn for_each_kv_entry_in_storage<Func>(
    &mut self,
    callback: Func
) -> Result<()> where
    Func: FnMut(KeyValuePair, u64) -> IndexOp, 

For each function for processing all KeyValuePair{} instances stored in the underlying storage.

The key value entries are processed in the following way:

• First we backup the current position of the underlying storage since it would otherwise be lost during scanning the entire storage file
• Next we seek to the start of the storage file
• Now in an infinite loop, during every iteration
  • We seek to the current position
  • We read a record using RiaKV::process_record
  • If the record read is not an error, we operate on it using the callback
  • In the case of an error
    • For simple EOF we break out of the loop
    • In the case of any other error, we return Err(err)
• Now if the callback is executed, the return value is used as follows:
  • For IndexOp::Insert the key value pair is inserted into the index
  • For IndexOp::Delete the key value pair is deleted from the index if it existed in the index before
  • For Index::Nop we do nothing a continue to the next iteration
  • For Index::End we break out of the loop
• When we exit from the loop, we seek back to the position we saved before entering into the loop
• We return Ok(())

Example

use libriakv::{RiaKV, IndexOp, ByteString, ByteStr};
use std::io;
use std::io::prelude::*;

// As used in the impl{} of RiaKV itself

fn load<F>(store: &mut RiaKV<F>) -> io::Result<()> where F: Read + Write + Seek {
    store.for_each_kv_entry_in_storage(|kv, position| {
        if kv.value.len() > 0 {
            IndexOp::Insert(kv, position)
        } else {
            IndexOp::Delete(kv, position)
        }
    })
}

// ...

fn find<F>(store: &mut RiaKV<F>, target: &ByteStr) -> io::Result<Option<(u64, ByteString)>>
    where F: Read + Write + Seek, {
     
    let mut found: Option<(u64, ByteString)> = None;

    store.for_each_kv_entry_in_storage(|kv, position| {
        if kv.key == target.to_vec() {
            found = Some((position, kv.value));
            IndexOp::End
        } else {
            IndexOp::Nop
        }
    })?;

   Ok(found)
}

pub fn load(&mut self) -> Result<()>

Loads all the key value entries from the underlying storage

pub fn get_at(&mut self, position: u64) -> Result<KeyValuePair>

Gets the KeyValuePair{} instance stored at the given position in the underlying storage.

pub fn get(&mut self, key: &ByteStr) -> Result<Option<ByteString>>

Gets the value for the given key.

Example

use libriakv::RiaKV;

let mut store = RiaKV::open_from_in_memory_buffer(5000);

store.insert(b"key", b"value").expect("insert");
store.get(b"key").expect("get").unwrap();

pub fn find(&mut self, target: &ByteStr) -> Result<Option<(u64, ByteString)>>

Finds the first KeyValueEntry{} corresponding to the given ByteStr key.

Note: Since this implementation is an append only, log structured store, deleted entries will always have corresponding entries.

Example

use libriakv::RiaKV;

let mut store = RiaKV::open_from_in_memory_buffer(5000);

store.insert(b"key", b"value").expect("insert");
store.find(b"key").expect("find").unwrap();

pub fn insert_but_ignore_index(
    &mut self,
    key: &ByteStr,
    value: &ByteStr
) -> Result<u64>

Inserts the given key value pair into the underlying storage and returns the position in the underlying storage file, it was written at. The index is not updated.

As mentioned before, the following layout is used for storing the key value pair:

┌────────────────┬────────────┬──────────────┬────────────────┐
│ crc32 checksum │ key length │ value length │ KeyValuePair{} │
└────────────────┴────────────┴──────────────┴────────────────┘

This method is intended to be used in the actual RiaKV::insert() implementation.

pub fn insert(&mut self, key: &ByteStr, value: &ByteStr) -> Result<()>

Inserts the given key value pair into the underlying storage and updates the index.

Example

use libriakv::RiaKV;

let mut store = RiaKV::open_from_in_memory_buffer(5000);
store.insert(b"key", b"value").expect("insert");

pub fn update(&mut self, key: &ByteStr, value: &ByteStr) -> Result<()>

Updates the value for the given key by inserting a duplicate entry into the storage and updating the index.

pub fn delete(&mut self, key: &ByteStr) -> Result<()>

Deletes the value for the given key by inserting a tombstone entry:

Equivalent implementation

use libriakv::RiaKV;

let mut store = RiaKV::open_from_in_memory_buffer(5000);
store.insert(b"key", b"").expect("delete");

impl<F> RiaKV<F> where
    F: Read + Write + Seek, 

pub fn load_index<R: Read>(&mut self, index_file: &mut R) -> Result<(), Error>

Loads the index from the given object implementing the Read trait. This done by deserializing the contents of the file using bincode::deserialize_from(reader) into a HashMap<ByteString, u64>.

The Read object is wrapped into a io::BufReader instance before reading the contents.

pub fn persist_index<W: Write>(&self, index_file: &mut W) -> Result<(), Error>

Writes the index into the given object implementing the Write trait, using bincode::serialize_into(writer, &self.index)

The Write object is wrapped into a io::BufWriter instance before writing the contents.

Trait Implementations

impl<F: Debug> Debug for RiaKV<F> where
    F: Read + Write + Seek, 

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.