Page 1 of 3 123 LastLast
Results 1 to 10 of 25

Thread: Data logger project

  1. #1
    Join Date
    Jun 2010
    Location
    Australia
    Posts
    3,734

    Default Data logger project

    This is an opening post to introduce a project I have been developing on dark stormy nights fueled by Jameson Irish Whiskey that I plan to make availble to members here at low cost, or build yourself. I will release the firmware etc. when completed. First I would like to share the concept and features and get some input to help finalise the design.

    So what is it?

    • USB powered data logger
    • Up to 4 x temperature sensor inputs on 3.5mm stereo jacks
    • Uses the Dallas DS18B20 sensor with up to 12 bit res (0.06 degrees C)
    • 1 x LPM head input (1mV per mW)
    • Build in backlit LCD


    Functionality:
    • Works stand-alone when USB powered, detecting connected sensors and adjusting screen automatically
    • Works with the free Parralax Data Log tool that integrates with MS Excel
    • Sample period adjustable via PC for 5, 10, 30 & 60 seconds


    This leaves you free to log to Excel easily and setup what graphing, trending and dashboard you want. I have been dialing in an old DPSS 532nm module, logging the LD TEC temp, heatsink temp and optical power which makes setup of the TEC PID control loop much easier! Make a small adjustment, go away, come back and look at the graphs. Easy.

    I plan to add option to select temp in C or F. Its all mounted in a small box the size of a pack of cards.

    If you dont want to log, but just the LCD data you can power it from any PC etc USB port or a plug pack USB charger like those available just about every electronics dept.

    By choosing the Dallas digital sensors there is no calibration required for temperature measurements, only a single 10 turn trimmer to calibrate the ADC reference for the LPM function, currently limited to 1023mW with a resolution of 1mW but I do plan to provide 3 or 4 hardware switched ranges, ie. 1, 2, 4 & 8W.

    Full details, pics, plans etc when its complete, but this will at least show some of the logging I have been doing and the dashboards I have found usefull. (next post)

    Edit: target price as a DIY build is $150 USD!!!!
    Last edited by dnar; 10-25-2013 at 22:56.
    This space for rent.

  2. #2
    Join Date
    Jun 2010
    Location
    Australia
    Posts
    3,734

    Default

    Screenshots of some logging I have been performing and various dashboards. The beauty of this design concept is your free to setup whatever graphing any number crunching you desire, and Excel lets you do lots of it.

    Click image for larger version. 

Name:	Logger 1.jpg 
Views:	11 
Size:	223.3 KB 
ID:	40635Click image for larger version. 

Name:	Logger 2.jpg 
Views:	10 
Size:	217.7 KB 
ID:	40636Click image for larger version. 

Name:	Logger 3.jpg 
Views:	13 
Size:	219.8 KB 
ID:	40637

    I hope to have the hardware in a prototype enclosure completed in a few days, pics to follow.
    This space for rent.

  3. #3
    Join Date
    Jun 2010
    Location
    Australia
    Posts
    3,734

    Default

    And one last logging session, this one is a beauty, it shows comparisons of test runs with various kP, kI & kD PID loop tuning parameters, as implemented in my main RGB projector with 3 x 92mm fans and the sensor on the optic side of the main optic table. This helped me understand the system dynamics and determine optimal loop tuning constants. I am a geek with a love of numbers, trends, logs etc. I am a lot of fun at parties and available for hire to bring a unique level of sophistication to your next event.

    Click image for larger version. 

Name:	Logger 4.jpg 
Views:	13 
Size:	203.3 KB 
ID:	40638
    This space for rent.

  4. #4
    Join Date
    Mar 2006
    Posts
    2,478

    Default

    Quote Originally Posted by dnar View Post
    I am a geek with a love of numbers, trends, logs etc. I am a lot of fun at parties and available for hire to bring a unique level of sophistication to your next event.
    Even supplies the Jameson's, apparently. Would the output be easy to parse for other tools that Excel? Sparkfun do a small logger (Logomatic) with an alternative 'Kwan' firmware that makes it better than the original, and I turned one into a GPS logger. No screen for output, but plain text and I think some other formats for data, and 8 ADC channels and a couple of serials. I only used one serial and one ADC to watch the battery, but it's been very useful to log cross-country running for years. It's one big lack is a display, but I plug it to USB and get the data out that way. I like plain text data, it's easier to parse and do other things with.

  5. #5
    Join Date
    Jun 2010
    Location
    Australia
    Posts
    3,734

    Default

    The serial stream is simple ASCII comma delimited text and you could modify the stream until the cows come home.
    This space for rent.

  6. #6
    Join Date
    Mar 2006
    Posts
    2,478

    Default

    Quote Originally Posted by dnar View Post
    The serial stream is simple ASCII comma delimited text and you could modify the stream until the cows come home.
    Unless the cows' udders are full of pure undiluted Jameson's they do not concern me, but the rest sounds cool. One thought: ethernet or standard RS-232 might be good to add. USB is common but a lot of engineering stuff still either sticks with RS-232, or goes with ethernet. I'm not sure if USB is actually bad from an engineering standpoint, but it has had more than its fair share of shaky reputation even in domestic PC situations.

    Edit: Actually, serial on 5V, like on little GPS modules. Wouldn't even need RS-232, as there are several tiny widgets out there barely bigger than the DB9 hardware for people who want long lines of it. But either way there's a lot of good using serial and AT commands to fetch stuff or change settings.
    Last edited by The_Doctor; 10-25-2013 at 23:27.

  7. #7
    Join Date
    Jun 2010
    Location
    Australia
    Posts
    3,734

    Default

    I am using serial over USB with the FTDI chip. Ethernet is easy, its already considered, in fact there is code space to implement a web server with custom web pages. As I am using an Arduino board, the options are endless, hence my choice. RS232, RS485, RS422, WiFi, Bluetooth, packet radio, all simple to add in.

    Hm, udders full of pure Jamesons... Good idea, let me work on that.
    This space for rent.

  8. #8
    Join Date
    Mar 2006
    Posts
    2,478

    Default

    Quote Originally Posted by dnar View Post
    I am using serial over USB with the FTDI chip. Ethernet is easy, its already considered, in fact there is code space to implement a web server with custom web pages. As I am using an Arduino board, the options are endless, hence my choice. RS232, RS485, RS422, WiFi, Bluetooth, packet radio, all simple to add in.

    Hm, udders full of pure Jamesons... Good idea, let me work on that.
    Really does sound good now. (Udderly fantastic, even). I think the Logomatic was related to the Arduino, but anyway... Web server could be really good because you can use the client with JavaScript to render graphs in any browser capable of it.

    I thought of bluetooth, but didn't mention it. Now that you did, it could be worth exploring, because it's fairly secure, can go to 100m with the right antenna, uses serial as a base 'profile'. I read somewhere that the EU loves Bluetooth, but the US barely used it. It's very good though, so that might change.

    Lots of options might mean a need to whittle it down to the vitals. Opinions will vary wildly on that, but my suggestion would be to keep ethernet with web, Bluetooth, and raw (TTL) serial, and USB. I suspect there are standard converters from those to most of the rest.

    How big will the board be? The one big strength of that Logomatic was that it's tiny.

    Incidentally, the Logomatic had a need to flush to SD card prior to power off to prevent SD card borkage. And if power failed suddenly, the card got borked anyway, and it could be permanent. I don't know if your board can benefit from the trick I used to fix that, but in case it can, here goes...
    (This is a bit vague as I can't find any of my documentation on it.) The Logomatic has a Li-ion power input and an LDO regulator for 3.3V, and a STOP line that when grounded caused flush to SD card followed by shutdown. I used an aerogel (or electric double layer) capacitor at 1 farad per amp of load current, connected from load side of 3.3V regulator to ground, and an LED with 1.8Vf in series with a 2K2 resistor from the Li-ion side of the regulator to trigger the stop line if the supply failed by pulling it to ground with 10K, and the few hundred milliseconds of supply in the cap were enough to safely flush to SD and prevent damage to it. Even if the code accidentally pulled the stop line low, no harm results, it would just light that LED. It's a bit fiddly to get the right balance of voltage so it stays up when power is there, regardless of the slight variations in Li-ion dischage voltage curve, and reliably triggers proper shutdown on breaking power supply. It works though, and in a projector if the supply is more constant, this idea is even easier to make reliable than it was for me. The main advantage of the scheme is that's it's passive, tiny, and very cheap, barring that cap. But what you spend on caps you more than save in time and effort. Bad shutdowns can and will happen, and this method absolutely protects the SD card, so you could justify making it inaccessible if you wanted to, for easier case design.
    Last edited by The_Doctor; 10-26-2013 at 01:19.

  9. #9
    Join Date
    Jun 2010
    Location
    Australia
    Posts
    3,734

    Default

    I can add SD card buffering easy, with Bluetooth and can monitor my projector temps on my Pebble watch. That's geeky, mid show 'hey look, my KTP crystal is temperature locked at 28.5 degrees' (looking at watch).

    You raise some good ideas. I have TTL serial available on the board now, you can drop in 232 or 485 transceiver to suit your needs.

    I am using the Freetronics 'Twenty Ten' version of the Arduino Duemilanove which I like as it includes enough on board proto area to mount the reference circuit and calibration trim pot plus the pull up required for the Dallas bus.

    http://www.freetronics.com/products/...n#.UmuUo-YZ7bU

    The board is small.

    Click image for larger version. 

Name:	5441230336_b026befe73_z.jpg 
Views:	8 
Size:	326.5 KB 
ID:	40639

    Freetronics have a model EtherTen that includes 10/100 Ethernet and an SD slot on the same size board while also being 100% compatible with Arduino (Its just like an Arduino with the Ethershield added, yet its a single board solution... I have one of them so that can be done next. No proto area, but that's OK.

    Click image for larger version. 

Name:	etherten-top-blue_large.jpg 
Views:	7 
Size:	70.4 KB 
ID:	40640

    When I release the open source code, anyone can tweak to suit their needs, add functions etc.
    Last edited by dnar; 10-26-2013 at 02:28.
    This space for rent.

  10. #10
    Join Date
    Mar 2006
    Posts
    2,478

    Default

    That single board idea might do for production. I'm not sure, but if it can, you might get to save a lot, and displace a lot of the difficulty into code. If your sight gets anything like mine as you get older, you won't regret doing that.

    About SD card buffering, would it be secure against power failure? As far as I know, there is always a chance that a break occurs during a write no matter how pre-emptive timing and code might be. And if it does, all bets are off. Hence my notion for a passive store of energy that guarantees to persist a while after the main power loss triggers the flush to card. Maybe better schemes exist, but SD cards don't have a lot of controller on them to play it safe. The details went over my head anyway so I decided a crude electronic fix would sort it best.

    That watch idea is cool, no doubt about it. Every time I wore a watch, or even had one, it broke down early somehow, no exceptions over several years. But if they'd survived and I'd got into the habit (and could actually see the tiny details well enough) that Pebble thinger sounds like a Good Thing. If a laser looks like going weird during a show it would be useful to know what can be done to prevent it before it happens.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •