Components - Arduino Web Logger

Diagram of Arduino Web Logger Component

Let's log ADAM-4017's AD value every 500 ms.
We are going to make 512 bytes long block every 1 minute and write that block to the SD card in Ethernet Shield.

Data Block Component.

• Used Flag: 4 bytes
  Set it to 0xaa55a55a.
• Time Stamp: 4 bytes
  This is only the sec part of POSIX time_t value.  We are not going to use the millisecond, thus we are going to multiply the value by 1000 (*1000).  It is 32 bit unsigned integer.
• Data: 4 bytes float value.  It is going to be sampled twice every second.
  1 minute measurement: 2 * 60 * 4 = 480 bytes
  Second is going to be offset.
  For example, The offset for second measurement in 27 second will be 8 + 27 * 8 + 4 = 228.
  Set the unmeasured values to -99.999.  When initializing the block set everything to -99.999.
• Backup: 24 bytes, set it to Null.

Data File.

Logging is going to be one File.

In FAT32, the biggest file size is 512 * 2³² = 2⁴¹ = 2,199,023,255,552 = 2GB.  Even the smallest SD card is bigger than 2GB.  With 2GB, we can log for 2,199,023,255,552 / 512 / 60 / 24 / 365 = 8,217 years.  Thus memory is going to be full approximately after 8,000 years so we don't have to worry about that.

Data inside File are going to be in time order, which will allow us to retrieve Data Block from certain time very quickly using the simplest Binary Search.  However, adjusting the RTC clock due to whatever reasons will screw this up.  ( Getting deeper into this is too much for this posting, so I am going to skip it. )

Web DB.

Arduino cannot hold all this log data like Browser might hope.  If the browser wants to look through a year long data "very quickly" it's impossible.  Thus, browser is going to keep log data in its database and once in a while connect with Arduino to get the most recent log and update its DB.