0.1.0 SparseArray

This commit is contained in:
rob tillaart 2022-07-18 15:54:11 +02:00
parent 081e744761
commit 54c9c28c65
16 changed files with 917 additions and 0 deletions

View File

@ -0,0 +1,13 @@
compile:
# Choosing to run compilation tests on 2 different Arduino platforms
platforms:
- uno
# - due
# - zero
# - leonardo
- m4
- esp32
- esp8266
# - mega2560
libraries:
- "SHT85"

View File

@ -0,0 +1,13 @@
name: Arduino-lint
on: [push, pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: arduino/arduino-lint-action@v1
with:
library-manager: update
compliance: strict

View File

@ -0,0 +1,17 @@
---
name: Arduino CI
on: [push, pull_request]
jobs:
runTest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.6
- run: |
gem install arduino_ci
arduino_ci.rb

View File

@ -0,0 +1,18 @@
name: JSON check
on:
push:
paths:
- '**.json'
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: json-syntax-check
uses: limitusus/json-syntax-check@v1
with:
pattern: "\\.json$"

View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2022-2022 Rob Tillaart
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -0,0 +1,116 @@
[![Arduino CI](https://github.com/RobTillaart/SparseArray/workflows/Arduino%20CI/badge.svg)](https://github.com/marketplace/actions/arduino_ci)
[![Arduino-lint](https://github.com/RobTillaart/SparseArray/actions/workflows/arduino-lint.yml/badge.svg)](https://github.com/RobTillaart/SparseArray/actions/workflows/arduino-lint.yml)
[![JSON check](https://github.com/RobTillaart/SparseArray/actions/workflows/jsoncheck.yml/badge.svg)](https://github.com/RobTillaart/SparseArray/actions/workflows/jsoncheck.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/RobTillaart/SparseArray/blob/master/LICENSE)
[![GitHub release](https://img.shields.io/github/release/RobTillaart/SparseArray.svg?maxAge=3600)](https://github.com/RobTillaart/SparseArray/releases)
# SparseArray
Arduino library for sparse arrays of floats.
TODO REDO DOCUMENTATION.
## Description
SparseArray is an **experimental** library to implement a one
dimensional sparse array of floats (a.k.a. vector) on an Arduino.
A sparse array is a mn array with mostly zeros and a low percentage
non-zero values.
The purpose of this library is efficient storage in memory.
The maximum array that can be represented is 65535 elements
with a theoretical maximum of 65535 non-zero elements.
(although that does not make sense due to overhead)
In practice the library limits this to 1000 non-zero elements.
Note: 255 elements would still fit in an UNO's 2K memory.
Relates to:
- https://github.com/RobTillaart/SparseMatrix
- https://github.com/RobTillaart/distanceTable
Note: this library is derived from SparseMatrix.
#### Implementation
The implementation is based on 2 arrays holding ``` x, value```
where value is float, and x is an uint16_t.
That are 6 bytes per element.
The number of elements that the sparse array object can hold are
given as parameter to the constructor.
If the space cannot be allocated the size is set to zero.
In the future other data types should be possible.
#### Performance
The elements are not kept sorted or indexed so optimizations might be
possible but are not investigated yet.
There is however a test sketch to monitor the performance of
the most important functions.
Accessing elements internally is done with a linear search,
which becomes (much) slower if the number of elements is increasing.
This means that although in theory there can be 65535 elements,
in practice a few 100 can already become annoyingly slow.
To keep performance a bit the library has a limit build in.
Check the .h file for **SPARSEARRAY_MAX_SIZE 1000**
## Interface
```cpp
#include "SparseArray.h"
```
### Constructor + meta
- **SparseArray(uint16_t size)** constructor.
Parameter is the maximum number of elements in the sparse array.
Note this number is limited to **SPARSEARRAY_MAX_SIZE 1000**.
If the space requested cannot be allocated size will be set to 0.
- **uint16_t size()** maximum number of elements.
If this is zero, a problem occurred with allocation happened.
- **uint16_t count()** current number of elements in the array.
Should be between 0 and size.
- **float sum()** sum of all elements ( != 0 ) in the array.
- **void clear()** resets the array to all zero's again.
### Access
- **bool set(uint16_t x, float value)** gives an element in the array a value.
If the value is set to zero, it is removed from the internal store.
Returns false if the internal store is full, true otherwise.
- **float get(uint16_t x)** returns a value from the array.
- **bool add(uint16_t x, float value)** adds value to an element in the array.
If needed a new internal element is created.
If the sum is zero, the element is removed from the internal store.
Returns false if the internal store is full, true otherwise.
- **void boundingSegment(uint16_t &minX, uint16_t &maxX)**
Returns the bounding box in which all values != 0 are located.
This can be useful for printing or processing the non zero elements.
## Future
- documentation
- test
- keep in sync with SparseMatrix where possible
- merge into one class hierarchy?
- dump should be in the class?
- or as static function...
- stream as param dump(Stream str, ...
#### ideas
- array { uint32_t, float }; for logging millis/micros + measurement
delta coding of time stamp? if it fit in 16 bit?
=> sounds like a class on its own.

View File

@ -0,0 +1,168 @@
//
// FILE: SparseArray.cpp
// AUTHOR: Rob Tillaart
// VERSION: 0.1.0
// DATE: 2022-07-17
// PURPOSE: Arduino library for sparse arrays of floats
// URL: https://github.com/RobTillaart/SparseArray
//
// HISTORY:
// 0.1.0 2022-07-17 initial version
// derives from SparseMatrix
#include "SparseArray.h"
SparseArray::SparseArray(uint16_t sz)
{
_count = 0;
_size = sz;
if ( _size > SPARSEARRAY_MAX_SIZE)
{
_size = SPARSEARRAY_MAX_SIZE;
}
_x = (uint16_t *) malloc(sz * sizeof(uint16_t));
_value = (float *) malloc(sz * sizeof(float));
// catch malloc error
if (_x && _value) return;
// if malloc error set size to zero.
_size = 0;
}
SparseArray::~SparseArray()
{
if (_x) free(_x);
if (_value) free(_value);
}
uint16_t SparseArray::size()
{
return _size;
}
uint16_t SparseArray::count()
{
return _count;
}
void SparseArray::clear()
{
_count = 0;
}
float SparseArray::sum()
{
float _sum = 0;
for (uint16_t i = 0; i < _count; i++)
{
_sum += _value[i];
}
return _sum;
}
bool SparseArray::set(uint16_t x, float value)
{
int32_t pos = findPos(x);
// existing element
if (pos > -1)
{
_value[pos] = value;
if (_value[pos] == 0.0) removeElement(pos);
return true;
}
// does not exist => new element ?
return newElement(x, value);
}
bool SparseArray::add(uint16_t x, float value)
{
int32_t pos = findPos(x);
// existing element
if (pos > -1)
{
_value[pos] += value;
if (_value[pos] == 0.0) removeElement(pos);
return true;
}
// does not exist => new element ?
return newElement(x, value);
}
float SparseArray::get(uint16_t x)
{
int32_t pos = findPos(x);
if (pos > -1)
{
return _value[pos];
}
return 0;
}
void SparseArray::boundingSegment(uint16_t &minX, uint16_t &maxX)
{
uint16_t _minx = 65535, _maxx = 0;
for (uint16_t i = 0; i < _count; i++)
{
if (_x[i] < _minx) _minx = _x[i];
if (_x[i] > _maxx) _maxx = _x[i];
}
minX = _minx;
maxX = _maxx;
}
//////////////////////////////////////////////////////
//
// PRIVATE
//
int32_t SparseArray::findPos(uint16_t x)
{
// linear search - not optimized.
for (uint16_t i = 0; i < _count; i++)
{
if (_x[i] == x)
{
return (int32_t)i;
}
}
return -1;
}
void SparseArray::removeElement(uint16_t pos)
{
_count--;
// move last element
// efficiency (keep sorted) is no requirement.
if (pos == _count) return;
_x[pos] = _x[_count];
_value[pos] = _value[_count];
}
bool SparseArray::newElement(uint16_t x, float value)
{
if (value == 0.0) return true;
if (_count >= _size) return false;
_x[_count] = x;
_value[_count] = value;
_count++;
return true;
}
// -- END OF FILE --

View File

@ -0,0 +1,65 @@
#pragma once
//
// FILE: SparseArray.h
// AUTHOR: Rob Tillaart
// VERSION: 0.1.0
// DATE: 2022-07-17
// PURPOSE: Arduino library for sparse arrays of floats
// URL: https://github.com/RobTillaart/SparseArray
//
#include "Arduino.h"
#define SPARSEARRAY_LIB_VERSION (F("0.1.0"))
#ifndef SPARSEARRAY_MAX_SIZE
#define SPARSEARRAY_MAX_SIZE 1000
#endif
class SparseArray
{
public:
SparseArray(uint16_t sz);
~SparseArray();
uint16_t size();
uint16_t count();
float sum();
void clear();
// returns false if no slots free
// could return # free slots?
bool set(uint16_t x, float value);
// adds value to element x,y
bool add(uint16_t x, float value);
float get(uint16_t x);
// returns two values between all values != 0 are located.
void boundingSegment(uint16_t &minX, uint16_t &maxX);
private:
uint16_t _size = 0;
uint16_t _count = 0;
uint16_t *_x = NULL; // support array's [0..65535]
float *_value = NULL;
// returns index of x, y if in set
// otherwise -1
int32_t findPos(uint16_t x);
// removes element at pos (from findPos)
// pre: count > 0
void removeElement(uint16_t pos);
// creates a new element if value != 0 and if there is room
bool newElement(uint16_t x, float value);
};
// -- END OF FILE --

View File

@ -0,0 +1,72 @@
//
// FILE: sparse_array_demo.ino
// AUTHOR: Rob Tillaart
// PURPOSE: demo
// URL: https://github.com/RobTillaart/SparseArray
#include "SparseArray.h"
// assume a sparse array of 100 floats
// filled with max of 20 non-zero elements == 20%
SparseArray sar(20);
void setup()
{
Serial.begin(115200);
Serial.println();
Serial.println(__FILE__);
Serial.println(sar.size());
Serial.println(sar.count());
Serial.println();
for (int i = 0; i < 20; i++)
{
uint8_t x = random(100); // 100 elements of max 255..
float value = EULER * PI * random(37);
bool b = sar.set(x, value); // check full
if (b) Serial.print('.');
}
Serial.println();
dump(100);
Serial.println();
Serial.print("NORMAL ARRAY: ");
Serial.println(100 * 4); // sizeof float
Serial.print("SPARSE ARRAY: ");
Serial.println(sar.size() * 6);
}
void loop()
{
}
void dump(uint8_t sx)
{
Serial.println();
Serial.print("DUMP\t");
Serial.print(sar.size());
Serial.print("\t");
Serial.print(sar.count());
Serial.print("\t");
Serial.print(sx);
Serial.print("\t");
Serial.println(sar.sum());
for (int x = 0; x < sx; x++)
{
if (x % 10 == 0) Serial.println();
Serial.print(sar.get(x));
Serial.print('\t');
}
Serial.println();
}
// -- END OF FILE --

View File

@ -0,0 +1,68 @@
//
// FILE: sparse_array_max.ino
// AUTHOR: Rob Tillaart
// PURPOSE: demo max size array
// URL: https://github.com/RobTillaart/SparseArray
#include "SparseArray.h"
// assume a sparse array of 2000 floats
// filled with max of 200 non-zero elements == 10%
SparseArray sar(200);
uint32_t start, stop;
uint32_t duration;
void setup()
{
Serial.begin(115200);
Serial.println();
Serial.println(__FILE__);
Serial.println(sar.size());
Serial.println(sar.count());
Serial.println();
delay(100);
for (int i = 0; i < 200; i++)
{
sar.set(random(2000), random(1000));
}
// this dump takes a while !
dump(2000);
}
void loop()
{
}
void dump(uint16_t sx)
{
Serial.println();
Serial.print("DUMP\t");
Serial.print(sar.size());
Serial.print("\t");
Serial.print(sar.count());
Serial.print("\t");
Serial.print(sx);
Serial.print("\t");
Serial.println(sar.sum());
for (uint16_t x = 0; x < sx; x++)
{
if (x % 10 == 0) Serial.println();
Serial.print(sar.get(x));
Serial.print('\t');
}
Serial.println();
}
// -- END OF FILE --

View File

@ -0,0 +1,14 @@
sparse_array_performance.ino
0.1.0
20
0
set 20x : 420
redo 20x : 424
full 20x : 568
add 20x : 444
get 20x : 300
0.00
sum 20x : 160
clr 20x : 4

View File

@ -0,0 +1,128 @@
//
// FILE: sparse_array_performance.ino
// AUTHOR: Rob Tillaart
// PURPOSE: performance measurement functions
// URL: https://github.com/RobTillaart/SparseArray
#include "SparseArray.h"
SparseArray sar(20);
uint32_t start, stop;
uint32_t duration;
void setup()
{
Serial.begin(115200);
Serial.println();
Serial.println(__FILE__);
Serial.println();
Serial.println(SPARSEARRAY_LIB_VERSION);
Serial.println(sar.size());
Serial.println(sar.count());
Serial.println();
delay(100);
start = micros();
for (int i = 0; i < 20; i++)
{
sar.set(i, 5);
}
stop = micros();
Serial.print("set 20x :\t");
Serial.println(stop - start);
delay(100);
start = micros();
for (int i = 0; i < 20; i++)
{
sar.set(i, 4);
}
stop = micros();
Serial.print("redo 20x :\t");
Serial.println(stop - start);
delay(100);
start = micros();
for (int i = 20; i < 40; i++)
{
sar.set(i, 5);
}
stop = micros();
Serial.print("full 20x :\t");
Serial.println(stop - start);
delay(100);
start = micros();
for (int i = 0; i < 20; i++)
{
sar.add(i, 5);
}
stop = micros();
Serial.print("add 20x :\t");
Serial.println(stop - start);
delay(100);
volatile float f;
start = micros();
for (int i = 0; i < 20; i++)
{
f = sar.get(i);
}
stop = micros();
Serial.print("get 20x :\t");
Serial.println(stop - start);
Serial.println(f);
delay(100);
start = micros();
f = sar.sum();
stop = micros();
Serial.print("sum 20x :\t");
Serial.println(stop - start);
delay(100);
start = micros();
for (int i = 0; i < 20; i++)
{
sar.clear();
}
stop = micros();
Serial.print("clr 20x :\t");
Serial.println(stop - start);
delay(100);
}
void loop()
{
}
void dump(uint16_t sx)
{
Serial.println();
Serial.print("DUMP\t");
Serial.print(sar.size());
Serial.print("\t");
Serial.print(sar.count());
Serial.print("\t");
Serial.print(sx);
Serial.print("\t");
Serial.println(sar.sum());
for (uint16_t x = 0; x < sx; x++)
{
if (x % 10 == 0) Serial.println();
Serial.print(sar.get(x));
Serial.print('\t');
}
Serial.println();
}
// -- END OF FILE --

View File

@ -0,0 +1,21 @@
# Syntax Colouring Map For SparseArray
# Data types (KEYWORD1)
SparseArray KEYWORD1
# Methods and Functions (KEYWORD2)
size KEYWORD2
count KEYWORD2
sum KEYWORD2
clear KEYWORD2
get KEYWORD2
set KEYWORD2
add KEYWORD2
boundingSegment KEYWORD2
# Constants (LITERAL1)
SPARSEARRAY_LIB_VERSION LITERAL1
SPARSEARRAY_MAX_SIZE LITERAL1

View File

@ -0,0 +1,23 @@
{
"name": "SparseArray",
"keywords": "Sparse,Array,memory",
"description": "Arduino library for sparse arrays of floats.",
"authors":
[
{
"name": "Rob Tillaart",
"email": "Rob.Tillaart@gmail.com",
"maintainer": true
}
],
"repository":
{
"type": "git",
"url": "https://github.com/RobTillaart/SparseArray.git"
},
"version": "0.1.0",
"license": "MIT",
"frameworks": "arduino",
"platforms": "*",
"headers": "SparseArray.h"
}

View File

@ -0,0 +1,11 @@
name=SparseArray
version=0.1.0
author=Rob Tillaart <rob.tillaart@gmail.com>
maintainer=Rob Tillaart <rob.tillaart@gmail.com>
sentence=Arduino library for sparse arrays of floats.
paragraph=
category=Data Processing
url=https://github.com/RobTillaart/SparseArray
architectures=*
includes=SparseArray.h
depends=

View File

@ -0,0 +1,149 @@
//
// FILE: unit_test_001.cpp
// AUTHOR: Rob Tillaart
// DATE: 2022-07-18
// PURPOSE: unit tests for SparseArray library
// https://github.com/RobTillaart/SparseArray
// https://github.com/Arduino-CI/arduino_ci/blob/master/REFERENCE.md
//
// supported assertions
// ----------------------------
// assertEqual(expected, actual)
// assertNotEqual(expected, actual)
// assertLess(expected, actual)
// assertMore(expected, actual)
// assertLessOrEqual(expected, actual)
// assertMoreOrEqual(expected, actual)
// assertTrue(actual)
// assertFalse(actual)
// assertNull(actual)
#include <ArduinoUnitTests.h>
#include "Arduino.h"
#include "SparseArray.h"
unittest_setup()
{
fprintf(stderr, "SPARSEARRAY_LIB_VERSION: %s\n", (char*) SPARSEARRAY_LIB_VERSION);
}
unittest_teardown()
{
}
unittest(test_constants)
{
assertEqual(1000, SPARSEARRAY_MAX_SIZE);
}
unittest(test_constructor)
{
SparseArray sar(10);
assertEqual(10, sar.size());
assertEqual(0, sar.count());
SparseArray sar2(1100);
assertEqual(1000, sar2.size());
}
unittest(test_set)
{
SparseArray sar(10);
assertEqual(10, sar.size());
for (int i = 0; i < 10; i++)
{
assertTrue(sar.set(i, 1.0 * i * i));
assertEqual(i, sar.count());
}
assertTrue(sar.set(13, 5));
assertFalse(sar.set(15, 5)); // don't fit any more...
// do not set new element to zero
sar.clear();
assertEqual(0, sar.count());
sar.set(1, 0);
assertEqual(0, sar.count());
}
unittest(test_get)
{
SparseArray sar(10);
assertEqual(10, sar.size());
for (int i = 0; i < 10; i++)
{
assertTrue(sar.set(i, 1.0 * i * i));
assertEqualFloat(1.0 * i * i, sar.get(i), 0.001);
}
}
unittest(test_sum)
{
SparseArray sar(10);
assertEqual(10, sar.size());
for (int i = 0; i < 10; i++)
{
assertTrue(sar.set(i, 10));
}
assertEqualFloat(100, sar.sum(), 0.0001);
}
unittest(test_add)
{
SparseArray sar(10);
assertEqual(10, sar.size());
for (int i = 0; i < 10; i++)
{
assertTrue(sar.add(i, 1.0 * i * i));
assertEqualFloat(1.0 * i * i, sar.get(i), 0.001);
}
for (int i = 0; i < 10; i++)
{
assertTrue(sar.add(i, 1.0 * i * i));
assertEqualFloat(2.0 * i * i, sar.get(i), 0.001);
}
for (int i = 0; i < 10; i++)
{
assertTrue(sar.add(i, -2.0 * i * i));
assertEqualFloat(0, sar.get(i), 0.001);
}
assertEqual(0, sar.count());
}
unittest(test_boundingSegment)
{
SparseArray sar(10);
assertEqual(10, sar.size());
// 10 element array - 6 random elements in the middle
for (int i = 0; i < 6; i++)
{
uint8_t x = random(5) + 3;
sar.set(x, random(37));
}
// random generator does not work
// assertEqual(6, sar.count());
assertEqual(1, sar.count());
uint16_t minX, maxX;
sar.boundingSegment(minX, maxX);
fprintf(stderr, "%d\t%d\n", minX, maxX);
}
unittest_main()
// -- END OF FILE --