View Poll Results: HAVE YOU EVER USED AN ILDA FORMAT THREE FILE

Voters
20. You may not vote on this poll
  • I have no idea what Formats I use. They just Load.

    5 25.00%
  • Yes, I've knowingly been sent a Format Three FILE

    5 25.00%
  • NO, I have not used a Format Three file.

    9 45.00%
  • Why Do I even care.

    1 5.00%
Page 2 of 8 FirstFirst 123456 ... LastLast
Results 11 to 20 of 79

Thread: Ilda format three

  1. #11
    Join Date
    Mar 2012
    Location
    Akron, Ohio USA
    Posts
    1,317

    Default

    It is a shame format 3 was never given a fair chance. The 16 byte header made it unnecessarily awkward to read any file that contained it because you could not use the same code and data structure to read it. You had to read part of the header and then determine if it was a 16 byte header or a 32 byte header, switch gears and then read the rest.

    The 16 byte header made no sense because it contained only the "ILDA" and binary "0003" followed by two 32 bit integers; one for the size of the data that follows in bytes and one for the quantity of elements. That might seem OK, but storing the size and the quantity is redundant. If you know one, you know the other. And using a 32 bit integer for quantity could cause a big problem. All the other sections use a 16 bit unsigned integer that can only count up to 65535 elements. A 32 bit number can count up to 4.3 billion elements. This number would be used for dynamic memory allocation to make room for the data that follows. If the number was an error your code might allocate up to 12GB of RAM. Not good.

    Like I have said many times before, format 3 was the proper answer for exactly what ILDA had been advertising on their website for many months before the release of the proposal in 2004. It was designed to extend the utility of the previous three section types. Format 3 works with formats 0 and 1.

    If an ILDA file reader that came before it had been written to skip over unrecognized stuff (as it is written in the spec) by looking for "ILDA" 000[0, 1 or 2] then it would skip format 3 and find the format 0 or 1 vector image that sits just beyond it.

    There is also another interesting side effect of using format 3 with 0 and 1. Formats 0 and 1 have palette indices in them. So every frame in a file could be 24 bit color (format 3) and palette color (format 2) and monochrome (by reading only the blanking byte).

    This would have been particularly useful for stand-alone projectors that read from fixed media like a CD ROM, SD card or EPROM. Some projectors might be fully analog modulated RGB capable of using the 24 bit color tables. Some projectors might have only 2 or 3 bits for each color (8 bits in total can define a palette of 256 colors). Some might be TTL (only 7 colors) and some might be monochrome.

    It would have been possible to create frames with 24 bit color and a well designed palette to produce exactly the colors that the artist wanted on all systems, all in the same file.

    Formats 4 & 5 do not work with 0 & 1. They are mutually exclusive. Although you might be able to store a mix of 0, 1, 4 & 5 in the same file, you cannot store a single frame that is both 0 or 1 and 4 or 5 at the same time. Skipping 4 or 5 will get you nothing. And using the same 32 byte header for 4 & 5 means that it suffers from the same defect as 0, 1 & 2. There is no way to know how big the section is without knowing what it contains.

    When I spoke to the top 3 executives of ILDA back in 2010 about all of this (Patrick Murphy, Dan Kohn, Tim Walsh) they all agreed that format 3 was a perfectly valid format for the reasons I stated above. They also saw no sense in the 16 byte header and they also thought that it should be fixed and added back into the specification.

    What happened to that idea?

    I hope this thread doesn't just dry up and blow away. Inquisitive minds want to know, while Dean sits there munching his popcorn.

    James.
    Last edited by james; 01-23-2015 at 16:40.
    Creator of LaserBoy!
    LaserBoy is free and runs in Windows, MacOSX and Linux.
    http://laserboy.org/code/LaserBoy_Current.zip
    http://theamerikans.org/LaserBoy

    http://laserboy.org/formatt/
    Ask me about my LaserBoy Correction Amp Kit for sale!

    Either do or do not do. There is no undo!

  2. #12
    Join Date
    Aug 2013
    Posts
    229

    Default

    I would recommend familiarizing yourself with the different file formats... As many have realized here, especially those starting out (like I did) on cheaper projectors, some projectors will ONLY display ONE format, and usually that format is limited to 0... My Goldenstar projector will display 0, 1, sometimes 4 and 5... The file 0 format is somewhat unique in the way it handles an x,y and a z axis for a third dimension; James of LaserBoy fame can clue you in more on this... There's a freeware program called Laser Show Composer that you can use as an ILDA file viewer: it's limited as well in what file formats it can show, but it will tell you what file format the ilda files are in. If you'r emore interested in the conversion process of ilda files, there's a thread I did where I was using LaserBoy to convert various file formats... Searching around should give ya an interesting ready for a few hours...

  3. #13
    Join Date
    Mar 2012
    Location
    Akron, Ohio USA
    Posts
    1,317

    Default

    I would be more than happy to further explain anything I wrote above.

    James.
    Creator of LaserBoy!
    LaserBoy is free and runs in Windows, MacOSX and Linux.
    http://laserboy.org/code/LaserBoy_Current.zip
    http://theamerikans.org/LaserBoy

    http://laserboy.org/formatt/
    Ask me about my LaserBoy Correction Amp Kit for sale!

    Either do or do not do. There is no undo!

  4. #14
    Join Date
    Jul 2008
    Location
    My momentum is too precisely determined :S
    Posts
    1,764

    Default

    Quote Originally Posted by VibrationsOfDoom View Post
    The file 0 format is somewhat unique in the way it handles an x,y and a z axis for a third dimension; James of LaserBoy fame can clue you in more on this...
    That's not really unique as format 4 is also 3D (and really similar to format 0).

    To break it down for you:

    Format
    0: 3D, palette (meaning every point has a number referring to a colour in a palette)
    1: 2D, palette
    2: palette table (the colours formats 0 and 1 use)
    3: the elusive format this thread is about, deprecated by Ilda after vandalisation by people with strong egos
    4: 3D, BGR (like RGB but reversed)
    5: 2D, BGR

    A format 0 or 1 file might start with a format 2 section (formats are per-section but usually not mixed up in one file). This format 2 section then has up to 256 colours in RGB format. The points in format 0 and 1 have a number which refers to a colour in the format 2 section. Most format 0/1 files don't have a format 2 section however and use a 64 colour palette.
    Due to the limitation to 256 colours (and a 64 colour palette being the most popular) and with the advancement of technology, the need for a true colour format rose. This became format 3, which is somewhat similar to format 2 as that it is a section with colours (RGB values) and it's before every geometry format section (0/1), but the difference is that it's no longer limited to 256 colours and that there should be as many colours as there are points in the next section, so every geometry section has a format 3 section before it where the colour of a point is determined by the corresponding entry in the format 3 section. This is the best way to ensure backwards compatibility: the format 0/1 points still have their number referring to the old format 2 palette and the Ilda standard clearly says that unrecognised formats should be ignored, so a properly implemented ilda file reader would be able to read format 3 files without problem as the true colour (format 3) section gets skipped while the points still have a reference to the colours in a format 2 (limited) palette.
    But there's one thing wrong with the format 3. All previous formats (0, 1 and 2) have a 32 byte header:

    1-4: 'I' 'L' 'D' 'A'
    5-8: 0 0 0 [format number: 0, 1 or 2]
    9-16: frame name (8 characters)
    17-24: company name (8 characters)
    25-26: total points/colours
    27-28: frame number
    29-30: total frames
    31: scanner head
    32: future use (should be 0)

    But a format 3 section header has 16 bytes! It's not known why Ilda chose to not continue the 32 byte tradition. Rumours say it's because of the strong ego of one of the collaborators. This makes reading a format 3 file problematic, as readers won't be able to tell how long the section is (by format 0/1/2 you can do that by reading bytes 25 and 26 in the header which are in another position in the 16 byte header of format 3). Actually, even if you could read it, there's no way to know how many bytes there are for each data point (a flaw which the other formats also have).

    Ilda then proceeded to create the formats 4 and 5 which again have a 32 byte header and are similar to formats 0 and 1, except the number in format 0/1 that referred to the palette colour is replaced by BGR values, so the 24 bit colour of each point is stored with the point itself and not in a separate section. I don't know when exactly they did this, but it was a long while ago and it's still not described on their website. Programmers who want to implement format 4/5 either need to be part of Ilda or reverse engineer existing files (which is quite easy when you know how to read format 0/1 files and know that 4/5 stores BGR values instead). And as is a continuous source of ire in James, 4/5 format files can't be read by 0/1 readers, while format 3 can be correctly read by readers who skip unknown sections and search for the next occurence of 'I' 'L' 'D' 'A' in the file (like my program, IldaViewer, does: when it encounters a format 3 file it just spams you with unrecognised format! messages but loads in the data correctly using the format 2 palette).


    All of this is on the PL wiki here: http://www.photonlexicon.com/wiki/in...tle=ILDA_files
    and here: http://www.photonlexicon.com/wiki/in...?title=Palette

  5. #15
    Join Date
    Mar 2012
    Location
    Akron, Ohio USA
    Posts
    1,317

    Default

    CMB, You are a champion!

    However you can define as many format 2 palettes as you want anywhere in the file.

    Once a format 2 palette is encountered in a file it stands for all format 0 or 1 frames that follow it in the natural order of the ILDA file, until another format 2 palette replaces it.

    Format 3 only works with the format 0 or 1 frame that immediately follows it.

    Format 2 palette has a one-to-many relationship with formats 0 or 1 and can only define up to 256 colors.

    Format 3 has a one-to-one relationship with a format 0 or 1 frame and must define as many colors as there are vertices in the following frame.

    With formats 0, 1, 2 & 3, formats 4 & 5 are unnecessary.

    James.
    Last edited by james; 01-23-2015 at 19:03.
    Creator of LaserBoy!
    LaserBoy is free and runs in Windows, MacOSX and Linux.
    http://laserboy.org/code/LaserBoy_Current.zip
    http://theamerikans.org/LaserBoy

    http://laserboy.org/formatt/
    Ask me about my LaserBoy Correction Amp Kit for sale!

    Either do or do not do. There is no undo!

  6. #16
    Join Date
    Jul 2008
    Location
    My momentum is too precisely determined :S
    Posts
    1,764

    Default

    Quote Originally Posted by james View Post
    However you can define as many format 2 palettes as you want anywhere in the file.

    Once a format 2 palette is encountered in a file it stands for all format 0 or 1 frames that follow it in the natural order of the ILDA file, until another format 2 palette replaces it.
    What you say is true, but I haven't encountered such a file before, and I doubt all file parsers like that. Many don't even support palettes, I can imagine half would crash if they encountered a format 2 somewhere in between. (IldaViewer has no trouble with in-between palettes, but doesn't allow to write them).

    Quote Originally Posted by james View Post
    With formats 0, 1, 2 & 3, formats 4 & 5 are unnecessary.

    James.
    Mathematically, yes, but I like formats 4/5 better. It's more logical to store RGB data within the point itself instead of a separate section. Less things to go wrong and it's easier to move points around (imagine an advanced dac having to do instream optimisation but then has to figure out colours, the only way is to store colour information within the points before optimisation and that's just better done beforehand, stupid example perhaps). Just a personal preference. It's true you lose backwards compability but the deed is long done and it's not like many readers look up ILDA000 in their files anyway (which is the only way to skip unfamiliar sections since even if you know the amount of data points there's no way to know how long the section is in bytes), especially since the standard specifies this is a bad way to look for sections (so of course that's how I do it, the probability of it going wrong is 1/(256^7) anyway).


    By the way, the wiki can always use some clarification if you feel inclined.

  7. #17
    Join Date
    Mar 2012
    Location
    Akron, Ohio USA
    Posts
    1,317

    Default

    I hope Robin isn't being told by other ILDA members that he broke The First Rule of Fight Club.

    He chimed in early with an announcement about the official ILDA file spec.

    I would love to see an official ILDA response to the issues outlined in this thread.

    I REALLY would!

    You are absolutely correct about formats 4 & 5. But like you said, it's done and out there.

    Has been for almost a decade.

    Nice to see that after almost all of the prominent developers have already incorporated it, ILDA is finally officiating it.

    It might be nice if format 4 & 5 files used the extension .ilda instead of .ild.

    That way you could sort them out easily.

    LB also knows how to look for ILDA000 but it only resorts to that if it gets lost in a busted file.

    I'm really looking forward to see where this thread goes.

    THANK YOU STEVE!

    James.
    Last edited by james; 01-23-2015 at 18:07.
    Creator of LaserBoy!
    LaserBoy is free and runs in Windows, MacOSX and Linux.
    http://laserboy.org/code/LaserBoy_Current.zip
    http://theamerikans.org/LaserBoy

    http://laserboy.org/formatt/
    Ask me about my LaserBoy Correction Amp Kit for sale!

    Either do or do not do. There is no undo!

  8. #18
    Join Date
    Mar 2012
    Location
    Akron, Ohio USA
    Posts
    1,317

    Default

    CMB, maybe we should get together to work on the wiki. You are welcome to use any of the gif image depictions of the various sections from here:

    http://laserboy.org/formatt

    But please document the source and don't copy them. Just use links to where they are.

    Click image for larger version. 

Name:	all_together.gif 
Views:	15 
Size:	57.4 KB 
ID:	46117

    James.
    Last edited by james; 01-23-2015 at 18:51.
    Creator of LaserBoy!
    LaserBoy is free and runs in Windows, MacOSX and Linux.
    http://laserboy.org/code/LaserBoy_Current.zip
    http://theamerikans.org/LaserBoy

    http://laserboy.org/formatt/
    Ask me about my LaserBoy Correction Amp Kit for sale!

    Either do or do not do. There is no undo!

  9. #19
    Join Date
    Mar 2012
    Location
    Akron, Ohio USA
    Posts
    1,317

    Default

    Quote Originally Posted by colouredmirrorball View Post
    What you say is true, but I haven't encountered such a file before, and I doubt all file parsers like that. Many don't even support palettes, I can imagine half would crash if they encountered a format 2 somewhere in between. (IldaViewer has no trouble with in-between palettes, but doesn't allow to write them).
    I have found this to be true as well with what little experience I have with other laser apps that read some flavor of ILDA.

    Proper support for format 2 palettes is rare.

    However, I hope you agree one cannot design a standard with limitations based on misinterpretations of developers at large.

    All one can do is write an accurate spec document and hope that developers get it right or ask for clarification.

    It's math. Good math is proof of itself.

    If you want some good examples of files that contain lots of palettes, import some format 4 or 5 or pick up your colors from a 24 bit per pixel bitmap in LaserBoy. It does a nice job of saving very accurate sections of formats 0, 1, 2 and 3.

    James.
    Last edited by james; 01-23-2015 at 19:14.
    Creator of LaserBoy!
    LaserBoy is free and runs in Windows, MacOSX and Linux.
    http://laserboy.org/code/LaserBoy_Current.zip
    http://theamerikans.org/LaserBoy

    http://laserboy.org/formatt/
    Ask me about my LaserBoy Correction Amp Kit for sale!

    Either do or do not do. There is no undo!

  10. #20
    Join Date
    Jul 2008
    Posts
    443

    Default

    Quote Originally Posted by james View Post
    LB also knows how to look for ILDA000 but it only resorts to that if it gets lost in a busted file.


    James.
    Really... I can take just about any file on my machine and slap on a .ild extension and try to open it in LB and get a total crash. That just shouldn't happen.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •