0.1.0 Student

This commit is contained in:
Rob Tillaart 2024-07-23 14:06:43 +02:00
parent da2e9fe077
commit 710ef84c52
16 changed files with 831 additions and 0 deletions

View File

@ -0,0 +1,34 @@
platforms:
rpipico:
board: rp2040:rp2040:rpipico
package: rp2040:rp2040
gcc:
features:
defines:
- ARDUINO_ARCH_RP2040
warnings:
flags:
packages:
rp2040:rp2040:
url: https://github.com/earlephilhower/arduino-pico/releases/download/global/package_rp2040_index.json
compile:
# Choosing to run compilation tests on 2 different Arduino platforms
platforms:
- uno
# - due
# - zero
# - leonardo
- m4
- esp32
- esp8266
# - mega2560
- rpipico
libraries:
- MultiMap
unittest:
# These dependent libraries will be installed
libraries:
- MultiMap

4
libraries/Student/.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1,4 @@
# These are supported funding model platforms
github: RobTillaart
custom: "https://www.paypal.me/robtillaart"

View File

@ -0,0 +1,13 @@
name: Arduino-lint
on: [push, pull_request]
jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: arduino/arduino-lint-action@v1
with:
library-manager: update
compliance: strict

View File

@ -0,0 +1,17 @@
name: Arduino CI
on: [push, pull_request]
jobs:
runTest:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.6
- run: |
gem install arduino_ci
arduino_ci.rb

View File

@ -0,0 +1,18 @@
name: JSON check
on:
push:
paths:
- '**.json'
pull_request:
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- name: json-syntax-check
uses: limitusus/json-syntax-check@v2
with:
pattern: "\\.json$"

View File

@ -0,0 +1,11 @@
# Change Log Student
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
## [0.1.0] - 2024-07-22
- initial version

21
libraries/Student/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2024-2024 Rob Tillaart
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

187
libraries/Student/README.md Normal file
View File

@ -0,0 +1,187 @@
[![Arduino CI](https://github.com/RobTillaart/Student/workflows/Arduino%20CI/badge.svg)](https://github.com/marketplace/actions/arduino_ci)
[![Arduino-lint](https://github.com/RobTillaart/Student/actions/workflows/arduino-lint.yml/badge.svg)](https://github.com/RobTillaart/Student/actions/workflows/arduino-lint.yml)
[![JSON check](https://github.com/RobTillaart/Student/actions/workflows/jsoncheck.yml/badge.svg)](https://github.com/RobTillaart/Student/actions/workflows/jsoncheck.yml)
[![GitHub issues](https://img.shields.io/github/issues/RobTillaart/Student.svg)](https://github.com/RobTillaart/Student/issues)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/RobTillaart/Student/blob/master/LICENSE)
[![GitHub release](https://img.shields.io/github/release/RobTillaart/Student.svg?maxAge=3600)](https://github.com/RobTillaart/Student/releases)
[![PlatformIO Registry](https://badges.registry.platformio.org/packages/robtillaart/library/Student.svg)](https://registry.platformio.org/libraries/robtillaart/Student)
# Student
Arduino library for Student or T-distribution math.
## Description
**Experimental**
Student is an experimental Arduino library to estimate the mean (average) of a population
based upon a (very) small sample (selection).
Goal is to calculate the interval for which there is a certain confidence e.g. 95%,
that the population mean lies between the estimated mean from the sample +- some intervalDelta.
The 0.1.x version of the library is limited to a maximum of 20 samples to keep RAM usage
relative low.
It uses a lookup table (LUT) for a sample size up to 20, and for five discrete confidence
levels 80%, 90%, 95%, 98% and 99% ==> size table = 20 x 5 = 100 floats.
The table encodes the 100 floats as uint16_t to save 50% of the RAM needed.
The library allows to calculate the interval after every addition until the internal buffer
of 20 samples is full. Note that at least 2 samples are needed to calculate the interval.
### StudentTable.h
The table is defined in the file **StudentTable.h** and generated from a spreadsheet.
If one need to extend the sample size, the file contains commented values for sample size 21-100.
One has to adjust **STUDENT_MAX_SIZE** in **StudentTable.h** too (max 255).
Note this will cost extra RAM.
If one wants to reduce the sample size / RAM, one can comment part of the table not needed.
One has to adjust **STUDENT_MAX_SIZE** too.
### Application example
If one wants to estimate the average temperature, one can measure it constantly and
calculate the average from all measurements. This may take up to 3600 samples per hour.
With the Student library one can measure the temperature once every 5 minutes.
This give 12 samples from which one can calculate the interval of the average temperature
with 95% confidence.
This takes far less samples and calculation time and might meet the insights you need.
Furthermore using this method can be ideal in an low power environment (e.g. remote,
battery powered sensor, with long sleep) to get the information you need.
### History
The T-distribution is developed by William Gosset, head experimental brewer, in 1908 while
working for Guinness Beer. His goal was to guard a constant quality of the beer brewed.
He needed a method to determine the quality of the raw materials like grains, malt and hops,
based on small samples from the fields.
For this he invented the t-distribution and published it under the name Student.
Therefore this distribution is also known as the Student distribution.
### Accuracy / precision
The version 0.1.x uses float for internal storage. This means precision is at most 6-7 digits.
The version 0.1.x lookup table has 20 x 5 values with 3 decimals, coded as an uint16_t (for RAM)
This allows about 3-4 digits precision for the found interval.
The 0.1.x version does not interpolate the confidence level (yet), and only support 5 distinct levels.
This interpolation (between 80-99) is planned to be implemented in the future.
If a non supported confidence level is used, the library will use 95%.
If you need only one confidence interval you could strip the lookup table to one column.
### Character
| parameter | name | ALT-code | char |
|:-----------:|:------:|:----------:|:-----:|
| mean | mu | ALT-230 | µ |
| stddev | sigma | ALT-229 | σ |
| CDF | phi | ALT-232 | Φ | ALT-237 for lower case
| plus minus | | ALT-0177 | ± |
- https://altcodesguru.com/greek-alt-codes.html
### Related
- https://en.wikipedia.org/wiki/Normal_distribution
- https://en.wikipedia.org/wiki/Student_distribution
- https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_probability/bs704_probability9.html
- https://github.com/RobTillaart/Gauss
- https://github.com/RobTillaart/Multimap
- https://github.com/RobTillaart/Statistic (more stat links there).
- https://github.com/RobTillaart/Student
## Interface
```cpp
#include Student.h
```
### Constructor + meta
- **Student()** constructor. 0.1.x has a fixed max sample size STUDENT_MAX_SIZE = 20.
- **uint8_t getSize()** returns STUDENT_MAX_SIZE == 20.
- **uint8_t getCount()** returns the number of samples added.
Returns value between 0 .. getSize().
- **void reset()** resets internal counter to zero.
### Add
- **bool add(float value)** adds one sample.
Returns false if internal buffer is full (count >= size).
- **bool add(float \*array, uint8_t size)** adds an array of samples.
Returns false if the internal buffer would "overflow".
### Math
- **float mean()** returns mean (average) of the samples added.
This is the estimated mean of the population from which the samples are taken.
- **float variance()** returns variance of the samples added.
- **float deviation()** returns standard deviation of the samples added.
- **float estimatedDeviation()** returns estimated deviation of the
estimated mean (based upon the samples).
### Interval
Confidence should be 80, 90, 95, 98 or 99.
The confidence level is not interpolated and incorrect values are replaced by 95%.
- **float intervalDelta(int confidence)** returns the delta to be added
oor subtracted to the mean to determine the confidence interval.
- **float meanLower(int confidence)** returns mean - intervalDelta.
- **float meanUpper(int confidence)** returns mean + intervalDelta.
## Future
#### Must
- documentation
- improve upon function names.
- test test test
#### Should
- t test example.
- optimize code, e.g. cache values for performance.
- optimize lookup table, PROGMEM for footprint?
- dynamic allocation for sizes > 20
- or derived classes, student30, student40, student50, student100?
- linear interpolation for values > 10 (performance?)
- add interpolation to **intervalDelta()** so confidence level (0.2.x)
could be any integer value from 80-99 (maybe even float?)
#### Could
- add examples
- add unit tests
- replace look up table with a formula? (performance drop!!)
- access function for internal array to access samples?
- template class instead of STUDENT_MAX_SIZE? (becomes different types).
- circular buffer for the samples? Running T-test?
#### Won't (unless requested)
## Support
If you appreciate my libraries, you can support the development and maintenance.
Improve the quality of the libraries by providing issues and Pull Requests, or
donate through PayPal or GitHub sponsors.
Thank you,

View File

@ -0,0 +1,11 @@
//
// FILE: Student.cpp
// AUTHOR: Rob Tillaart
// VERSION: 0.1.0
// PURPOSE: Arduino library for Student or T-distribution math.
// DATE: 2024-07-22
// placeholder
// -- END OF FILE --

142
libraries/Student/Student.h Normal file
View File

@ -0,0 +1,142 @@
#pragma once
//
// FILE: Student.h
// AUTHOR: Rob Tillaart
// VERSION: 0.1.0
// PURPOSE: Arduino library for Student or T-distribution math.
// DATE: 2024-07-22
#include "Arduino.h"
#include "StudentTable.h"
#define STUDENT_LIB_VERSION (F("0.1.0"))
class Student
{
public:
Student()
{
_size = STUDENT_MAX_SIZE;
_count = 0;
}
uint8_t getSize()
{
return _size;
}
uint8_t getCount()
{
return _count;
}
void reset()
{
_count = 0;
}
//
// ADD
//
bool add(float value)
{
if (_count >= _size) return false;
_value[_count++] = value;
return true;
}
bool add(float *array, uint8_t size)
{
if (_count + size > _size) return false;
for (int i = 0; i < size; i++)
{
add(array[i]);
}
return true;
}
//
// MATH
//
float mean()
{
float sum = 0;
for (int i = 0; i < _count; i++)
{
sum += _value[i];
}
return sum / _count;
}
float variance()
{
if (_count < 2) return NAN;
float m = mean();
float var = 0;
for (int i = 0; i < _count; i++)
{
var += (_value[i] - m)*(_value[i] - m);
}
var /= (_count - 1);
return var;
}
float deviation()
{
if (_count < 2) return NAN;
float var = variance();
return sqrt(var);
}
float estimatedDeviation()
{
if (_count < 2) return NAN;
float dev = deviation();
return dev / sqrt(_count);
}
//
// INTERVAL
//
float intervalDelta(int confidence)
{
if (_count < 2) return NAN;
// degrees freedom
int df = _count - 1;
int idx = 2; // 95% default
if (confidence == 80) idx = 0;
if (confidence == 90) idx = 1;
if (confidence == 98) idx = 3;
if (confidence == 99) idx = 4;
// interpolation possible
float t = StudentLUT[df][idx] * 0.001;
return estimatedDeviation() * t;
}
float meanLower(int confidence)
{
return mean() - intervalDelta(confidence);
}
float meanUpper(int confidence)
{
return mean() + intervalDelta(confidence);
}
private:
uint8_t _size;
uint8_t _count;
float _mean = 0;
float _value[STUDENT_MAX_SIZE];
};
// -- END OF FILE --

View File

@ -0,0 +1,159 @@
#pragma once
//
// FILE: StudentTable.h
// AUTHOR: Rob Tillaart
// VERSION: 0.1.0
// PURPOSE: Arduino library for Student or T-distribution math.
// DATE: 2024-07-22
// lookup table, do not change (unless you know what you do).
//
// spreadsheet = ROUND(1000 * T.INV.2T(C$4;$B7))
//
// x = degrees of freedom
// y = confidence level 0.1, 0.05, 0.025, 0.01, 0.005 (single ended)
// values are multiplied by 1000, to save RAM.
//
// command line overrule possible
#ifndef STUDENT_MAX_SIZE
#define STUDENT_MAX_SIZE (20)
#endif
// uses 200 bytes of RAM
uint16_t StudentLUT[STUDENT_MAX_SIZE][5]
{
// 80% 90% 95% 98% 99%
//---------------------------------
{ 3078, 6314, 12706, 31821, 63657}, // n = 1
{ 1886, 2920, 4303, 6965, 9925},
{ 1638, 2353, 3182, 4541, 5841},
{ 1533, 2132, 2776, 3747, 4604},
{ 1476, 2015, 2571, 3365, 4032},
{ 1440, 1943, 2447, 3143, 3707},
{ 1415, 1895, 2365, 2998, 3499},
{ 1397, 1860, 2306, 2896, 3355},
{ 1383, 1833, 2262, 2821, 3250},
{ 1372, 1812, 2228, 2764, 3169},
{ 1363, 1796, 2201, 2718, 3106},
{ 1356, 1782, 2179, 2681, 3055},
{ 1350, 1771, 2160, 2650, 3012},
{ 1345, 1761, 2145, 2624, 2977},
{ 1341, 1753, 2131, 2602, 2947},
{ 1337, 1746, 2120, 2583, 2921},
{ 1333, 1740, 2110, 2567, 2898},
{ 1330, 1734, 2101, 2552, 2878},
{ 1328, 1729, 2093, 2539, 2861},
{ 1325, 1725, 2086, 2528, 2845} // n = 20
};
/*
{ 1323, 1721, 2080, 2518, 2831}, // n = 21
{ 1321, 1717, 2074, 2508, 2819},
{ 1319, 1714, 2069, 2500, 2807},
{ 1318, 1711, 2064, 2492, 2797},
{ 1316, 1708, 2060, 2485, 2787},
{ 1315, 1706, 2056, 2479, 2779},
{ 1314, 1703, 2052, 2473, 2771},
{ 1313, 1701, 2048, 2467, 2763},
{ 1311, 1699, 2045, 2462, 2756},
{ 1310, 1697, 2042, 2457, 2750}, // n = 30
{ 1309, 1696, 2040, 2453, 2744},
{ 1309, 1694, 2037, 2449, 2738},
{ 1308, 1692, 2035, 2445, 2733},
{ 1307, 1691, 2032, 2441, 2728},
{ 1306, 1690, 2030, 2438, 2724},
{ 1306, 1688, 2028, 2434, 2719},
{ 1305, 1687, 2026, 2431, 2715},
{ 1304, 1686, 2024, 2429, 2712},
{ 1304, 1685, 2023, 2426, 2708},
{ 1303, 1684, 2021, 2423, 2704}, // n = 40
{ 1303, 1683, 2020, 2421, 2701},
{ 1302, 1682, 2018, 2418, 2698},
{ 1302, 1681, 2017, 2416, 2695},
{ 1301, 1680, 2015, 2414, 2692},
{ 1301, 1679, 2014, 2412, 2690},
{ 1300, 1679, 2013, 2410, 2687},
{ 1300, 1678, 2012, 2408, 2685},
{ 1299, 1677, 2011, 2407, 2682},
{ 1299, 1677, 2010, 2405, 2680},
{ 1299, 1676, 2009, 2403, 2678}, // n = 50
{ 1298, 1675, 2008, 2402, 2676},
{ 1298, 1675, 2007, 2400, 2674},
{ 1298, 1674, 2006, 2399, 2672},
{ 1297, 1674, 2005, 2397, 2670},
{ 1297, 1673, 2004, 2396, 2668},
{ 1297, 1673, 2003, 2395, 2667},
{ 1297, 1672, 2002, 2394, 2665},
{ 1296, 1672, 2002, 2392, 2663},
{ 1296, 1671, 2001, 2391, 2662},
{ 1296, 1671, 2000, 2390, 2660}, // n = 60
{ 1296, 1670, 2000, 2389, 2659},
{ 1295, 1670, 1999, 2388, 2657},
{ 1295, 1669, 1998, 2387, 2656},
{ 1295, 1669, 1998, 2386, 2655},
{ 1295, 1669, 1997, 2385, 2654},
{ 1295, 1668, 1997, 2384, 2652},
{ 1294, 1668, 1996, 2383, 2651},
{ 1294, 1668, 1995, 2382, 2650},
{ 1294, 1667, 1995, 2382, 2649},
{ 1294, 1667, 1994, 2381, 2648}, // n = 70
{ 1294, 1667, 1994, 2380, 2647},
{ 1293, 1666, 1993, 2379, 2646},
{ 1293, 1666, 1993, 2379, 2645},
{ 1293, 1666, 1993, 2378, 2644},
{ 1293, 1665, 1992, 2377, 2643},
{ 1293, 1665, 1992, 2376, 2642},
{ 1293, 1665, 1991, 2376, 2641},
{ 1292, 1665, 1991, 2375, 2640},
{ 1292, 1664, 1990, 2374, 2640},
{ 1292, 1664, 1990, 2374, 2639}, // n = 80
{ 1292, 1664, 1990, 2373, 2638},
{ 1292, 1664, 1989, 2373, 2637},
{ 1292, 1663, 1989, 2372, 2636},
{ 1292, 1663, 1989, 2372, 2636},
{ 1292, 1663, 1988, 2371, 2635},
{ 1291, 1663, 1988, 2370, 2634},
{ 1291, 1663, 1988, 2370, 2634},
{ 1291, 1662, 1987, 2369, 2633},
{ 1291, 1662, 1987, 2369, 2632},
{ 1291, 1662, 1987, 2368, 2632}, // n = 90
{ 1291, 1662, 1986, 2368, 2631},
{ 1291, 1662, 1986, 2368, 2630},
{ 1291, 1661, 1986, 2367, 2630},
{ 1291, 1661, 1986, 2367, 2629},
{ 1291, 1661, 1985, 2366, 2629},
{ 1290, 1661, 1985, 2366, 2628},
{ 1290, 1661, 1985, 2365, 2627},
{ 1290, 1661, 1984, 2365, 2627},
{ 1290, 1660, 1984, 2365, 2626},
{ 1290, 1660, 1984, 2364, 2626} // n = 100
120 1289 1658 1980 2358 2617
10000 1282 1645 1960 2327 2576 // effectively infinity.
*/
// -- END OF FILE --

View File

@ -0,0 +1,65 @@
//
// FILE: Student_test.ino
// AUTHOR: Rob Tillaart
// PURPOSE: demo
#include "Student.h"
Student S;
void setup(void)
{
Serial.begin(115200);
Serial.println(__FILE__);
Serial.print("STUDENT_LIB_VERSION: ");
Serial.println(STUDENT_LIB_VERSION);
Serial.println();
S.reset();
S.add(12.4);
S.add(11.8);
S.add(12.0);
S.add(11.7);
S.add(12.1);
S.add(12.3);
S.add(11.9);
S.add(11.6);
S.add(11.9);
S.add(12.3);
// for (int i = 0; i < 10; i++)
// {
// S.add(i);
// }
Serial.print("Size:\t");
Serial.println(S.getSize());
Serial.print("Count:\t");
Serial.println(S.getCount());
Serial.print("mean:\t");
Serial.println(S.mean(), 3);
Serial.print("var:\t");
Serial.println(S.variance(), 3);
Serial.print("dev:\t");
Serial.println(S.deviation(), 3);
Serial.print("E dev:\t");
Serial.println(S.estimatedDeviation(), 3);
Serial.print("MeanLow:\t");
Serial.println(S.meanLower(95), 3);
Serial.print("MeanUp:\t");
Serial.println(S.meanUpper(95), 3);
Serial.println("\ndone...");
}
void loop(void)
{
}
// -- END OF FILE --

View File

@ -0,0 +1,26 @@
# Syntax Colouring Map For Student
# Data types (KEYWORD1)
Student KEYWORD1
# Methods and Functions (KEYWORD2)
getSize KEYWORD2
getCount KEYWORD2
reset KEYWORD2
add KEYWORD2
mean KEYWORD2
variance KEYWORD2
deviation KEYWORD2
estimatedDeviation KEYWORD2
intervalDelta KEYWORD2
meanLower KEYWORD2
meanUpper KEYWORD2
# Constants (LITERAL1)
STUDENT_LIB_VERSION LITERAL1

View File

@ -0,0 +1,23 @@
{
"name": "Student",
"keywords": "probability, statistic",
"description": "Arduino library for Student or T-distribution math.",
"authors":
[
{
"name": "Rob Tillaart",
"email": "Rob.Tillaart@gmail.com",
"maintainer": true
}
],
"repository":
{
"type": "git",
"url": "https://github.com/RobTillaart/Student.git"
},
"version": "0.1.0",
"license": "MIT",
"frameworks": "*",
"platforms": "*",
"headers": "Student.h"
}

View File

@ -0,0 +1,12 @@
name=Student
version=0.1.0
author=Rob Tillaart <rob.tillaart@gmail.com>
maintainer=Rob Tillaart <rob.tillaart@gmail.com>
sentence=LArduino library for Student or T-distribution math.
paragraph=probability, statistic.
category=Data Processing
url=https://github.com/RobTillaart/Student
architectures=*
includes=Student.h
depends=

View File

@ -0,0 +1,88 @@
//
// FILE: unit_test_001.cpp
// AUTHOR: Rob Tillaart
// DATE: 2024-07-22
// PURPOSE: unit tests for the Student library
// https://github.com/RobTillaart/Student
// https://github.com/Arduino-CI/arduino_ci/blob/master/REFERENCE.md
//
// supported assertions
// ----------------------------
// assertEqual(expected, actual)
// assertNotEqual(expected, actual)
// assertLess(expected, actual)
// assertMore(expected, actual)
// assertLessOrEqual(expected, actual)
// assertMoreOrEqual(expected, actual)
// assertTrue(actual)
// assertFalse(actual)
// assertNull(actual)
#include <ArduinoUnitTests.h>
#include "Arduino.h"
#include "Student.h"
unittest_setup()
{
fprintf(stderr, "STUDENT_LIB_VERSION: %s\n", (char *) STUDENT_LIB_VERSION);
}
unittest_teardown()
{
}
unittest(test_constants)
{
assertEqual(20, STUDENT_MAX_SIZE);
}
unittest(test_constructor)
{
Student S;
assertEqual(20, S.getSize());
assertEqual(00, S.getCount());
}
unittest(test_all)
{
Student S;
S.reset();
assertEqual(0, S.getCount());
S.add(12.4);
S.add(11.8);
S.add(12.0);
S.add(11.7);
S.add(12.1);
S.add(12.3);
S.add(11.9);
S.add(11.6);
S.add(11.9);
S.add(12.3);
assertEqual(10, S.getCount());
assertEqualFloat(12.0, S.mean(), 0.001);
assertEqualFloat(0.073, S.variance(), 0.001);
assertEqualFloat(0.271, S.deviation(), 0.001);
assertEqualFloat(0.086, S.estimatedDeviation(), 0.001);
assertEqualFloat(11.809, S.meanLower(95), 0.001);
assertEqualFloat(12.191, S.meanUpper(95), 0.001);
}
unittest_main()
// -- END OF FILE --