[SOLVED] XML --> FRW converter, Somebody who could help me?

  • So around 8 years ago I started experimenting with something which turned out to one of my biggest projects ever: A remake of Efteling's fountainshow "Aquanura" in RCT3 as precise as possible. There are/were a lot of different aspects to this project. Like the layout of the fountains with firework launchers created with CSO, the entire area around the like creating with CSO objects and mix of foliage to create the Efteling feeling around it, and most importantly the show itself using AdvancedFireworkEditor to create all the particle effects for the fountains, and putting it together sing MixMaster where I use CFRs to apply animation to certain fountains.


    Now because I want to recreate the show is precisely as possible, creating the fountains themselves comes along with a lot of complex work to apply the right color schemes to the fountain effects. All this has to be done by one by one adjusting time values and dragging boxes with certain color settings in the AdvancedFireworkEditor. This proces has become so much time consuming and complex that around 2 a 3 years ago, I lost all motivation and inspiration (in combination with everything from the RCT3 community falling apart).


    A little time ago I decided that I probably wouldnt finish it anyway, so I uploaded videos of all the scenes in the state as they are now, just to show how far I've come and mentally I've given up a bit on finishing it.


    Full playlist: https://www.youtube.com/playli…LZM3xua83SWoUKC5n7gOhH5-V

    Full project history: https://www.shyguysworld.com/index.php?topic=11788.0


    Now personally, It still bothers me a bit that I have never finished it because I've put so much effort in it. So I'm investigating one last option to give me a boost of motivation and finding a way to make creating the show much easier for me.


    Now the problem is that in the AdvancedFireworkEditor, I've to adjust everything by hand one by one. But I could calculate everything what I have to adjust based on timing data en color data (which i can easily get for myself). So the idea what I had is that instead of adjusting the effects ingame, I could somehow let the effects automatically generate to obtain certain effects for a specific moment in the show.


    shot0027-jpg.140722


    Now an particle effect (.frw-file) is build up like a tree structure with parent-child relations, and every point in this structure has its own data. In the picture shown above you see a bit of such an particle effect with a tree structure.


    Now I would have the possibilities and knowledge to generate xml files which describes such tree structures with all the data for each point, and let that data be determined by calculations based on input values and output I want.

    The only problem is that I've absolutely 0 knowledge in how this can be turned into the file format of .frw files so it can be used in RCT3.


    This is were I ask for someone's help. If there is someone who could help me with creating a program which can convert such an xml file with certain described data to a .frw-file so i have the particle effects I need ingame, I can do things much more easily and time-efficient, so I could get the motivation again to finish this big project and finish it sooner as well.


    Is there someone who could help me?

  • So unfortunately there was no-one to response on this. So in one last effort to decide if it's worth time to go into it or not, i have some questions I hope someone can answer (I hope Markus can answer some but cant find a way to ask him directly xD)


    -What is the best strategy to figure out a file structure with its data and how to read en overwrite this data. In this case for RCT3's .frw files
    I mostly mean here: where to start, what kind of information and skills do I need (cause in terms of programming I have zero experience), on what kind of information do I have to google.
    (I have tried looking into the file with a HeX editor but that was not really successful)


    -What is already known about the file structure of .frw files? Has someone already stored some information on this file-type and is able to share this information


    -Too fully debug the filestructure and be able to make a program where I could transfer data from an for example xml file (in specific form) to a .frw file, how much time would it approximately take for a noobie like me?

    I hope someone can at least give me some answers so I can consider if this is worth trying to be able to finish the project more easily, or if I'd better call it quit for this project.

  • Hello Mennoo,


    you can't write me directly because I have blocked all PM. Forum posts are better, because all users can profit from them if they have the same problem.


    I will take a look into .frw files to see which basic data structure is used. Depending on the result I will inform you, if modifications are possible, how difficult they are and how long it could take for a newbie.

  • Thank you Markus for the answer!

    Little information I just thought about: Some years ago on the old vPyro forum (forum specifically for RCT3 firework shows), someone posted his progress on an xtreme firework editor. the same as the ingame advanced firework editor, but with some extra features like:
    -ability to copy-paste data from one emitter to another (emitter is one bulb in that tree structure of an particle effect)
    -values without increments. the ingame interface works with increments like lifetimes for example can only be increased with increments of 0.05s and rotation speed only with increments of pi/25. He worked with ability to type in any value without those increment restriction.

    -larger view of that tree-structure


    So this suggest that reading and changing data in the file is possible.

    One thing he mentioned at that time which is kind of important: He said that he could only make it work when you load in an already existing .frw file and change it, and that the program could generate a new frw file from scretch. Now what I've seen in the HeX editor, there is a lot of data at the top which seem to have no influence on the data in the tree-structure, so this might explain it.
    However, i think this is easily solvable if you just if a default .frw file with the sole basis of a tree structure from which you can build and change the tree-structure as you wish.

    Unfortunately, he never released his xtreme firework editor, nor is anything visible from it nowadays since vPyro has been offline for a long time now

  • Hello Mennoo,


    I found that .frw files use the so called ".dat" structure (not the ".ovl" structure).

    .dat = park files, saved structures, firework effects

    .ovl = scenery, flat rides, coasters, cars, tracks


    My knowledge is mainly on .ovl files. In the beginning CTR ride were generated by using an already existing .ovl and modified it. Later the program CTR_Creator creates .ovl files from scratch.


    Unfortunately there is no program that creates .dat (or converts .xml to .dat). Basically it is possible, and creating a .dat structure is easier than creating a .ovl structure. My prognosis is about 6 months for a newbie like you to write such a program, with a lot of learning and try & errors. I am sorry, but I think it is not worth the efforts for just one fountainshow "Aquanura".


    I remember the program "Park Cleanup", which tried to repair park files (.dat) and reduced the file size. This means that .dat files were successfully modified. Maybe you can contact the user who wrote the program "Park Cleanup".


    Regarding "vPyro" you could try the WayBackMachine:

    https://web.archive.org/web/20…/http://forums.vpyro.com/ (Overview 2016)

    and

    https://web.archive.org/web/20…/http://forums.vpyro.com/ (last save I found)

    Some years ago I started to include a .dat file analyzer (beta) in CTR_Creator. As far as I remember I never finished or published it. But maybe I will continue it someday. Then I will inform you.

  • Thank you for the answer Markus.

    I already kind of knew that frw files have the .dat file structure, you can already read them with the ParkCleanup.
    I still have daily contact with Joeywp (creator of the Park Cleanup). Unfortunately, due to his daily job, scripting-work for other hobbies and lack of interest in RCT3 there isnt much he can help me with anymore. Furthermore, he lost some of his backups for ParkCleanup and information on file structures which makes it even more difficult to step into it.


    He only found lately an old, very basic RCT3 archive file reader which he has shown me: link to github

    But when I try to read and understand it I have absolutely no clue what I'm reading and how the script actually works. And that is basically my problem, I have absolutely no clue how decompiling such a file structure as a .dat file actually works and where to start with that to even understand how it works.

    Furthermore, When you load in an frw file in the ParkCleanup 2, you get most of the information with all the ParticleBaseDesc, multiple ParticleEmitterDesc and ParticleDesc. but I'm mostly interested in the data that is inside those descs. What I'm mainly aiming for is somehow to write an xml file which mimic the ingame advanced firework edittor, which basically tells which ParticleEmitterDesc and ParticleDesc parts has to be inside the frw file with certain values, and then writes that to an frw file so i dont have to adjust everything one-by-one by hand.
    This is basically what I've asked Joey as well if he could help me with that (around 1 a 2 years ago now), but for obvious reasons which I can completely understand, he isn't interested in doing so.
    Asking it here was kind of my last hope to see if there could still be something possible (because inside I kept feeling kind of sad to have put a lot of effort for several years in this project, just to never finish it). But knowing it would take me more than 6 months isn't a hopeful sign either.


    Anyway, thank you for the quick answer Markus!


    PS: I tried the webarchive already long time ago, but couldnt find the specific topic, and even so, the only way you could have contacted him anyways was with PM on vPyro (which i did once when it was still online)

  • Your Github link is quite interesting. It is Javascript (which I don't know), but it is very similar to C++ (which I use for programming). Therefore I understand almost everything.


    If this script is really all that is included in .dat structures, then these files are quite simple and less time consuming than I thought. This will help me to finish the .dat file analyzer (beta) in CTR_Creator, which I mentioned in my last post. Thanks a lot! You will be the first person to receive a preliminary version for testing. Then I would be glad if we could work together to do further development for modifying .frw files or even write them from scratch.

  • okay, so little update from my side now:


    Joey managed to further help me with finding his c++ files from ParkCleanup2, and later-on with the a site I needed most for understanding the filestructure: http://belgabor.vodhin.org/format/


    With this and some functions Joey already had written to be able to read the file in binary, I managed to write my own C++ program which is now able to read all data in an FRW file, save it in variables and to print the entire filestructure out in the console. Only step now for me is to find out and experiment how to write down data back to a frw file, but I'm sure I can manage that now.

  • Congratulations to your progress. This is a big step forward, not just a "little update".


    I am quite surprised that Vodhin's website is still up and running. In the good old days it was a great source for CSO knowledge. And nowadays it still helps you to understand the .dat file structure. RCT3 is not dead yet.

  • Mennoo_

    Hat den Titel des Themas von „XML --> FRW converter, Somebody who could help me?“ zu „[SOLVED] XML --> FRW converter, Somebody who could help me?“ geändert.
  • Alright, another "Big step forward" update then.


    The last 2 days i have finished the last details on the reading part, and more importantly: finished the writing back part.
    Now I can complete read and write back frw files. and I've already executed a test where i generate 44 different frw files with each a different time-span.

    The result:



    What you see here is that i have auto-generated 44 different frw files for the little white light-bulbs below each individual fountain. These light bulbs are constructed in such a way that they go on as intended with the choreographics in mixmaster, but all go out again at exactly the same time.

    It's obviously that the test is quite succesfull. It's not 100% perfect tho. To use the files inside mixmaster works perfectly fine, but when you want to open the file inside the advanced firework editor (where you would normally edit frw files inside RCT3), the fily cannot be opend cause it will cause a error with msgbox: "... File might be corrupted?...". It doesnt crash the game tho so that is a good thing. Why the AFE cant open the files might be cause i cant save them with it's footer (which is a uint32 value, probably a checksum which might be something of the kind crc32, which joey told me. The algorithm behind this value is unknown however).

    So now i have everything working which i need, so i can further focus on auto-generating more frw files which i can use inside the show



    EDIT: Just did some checking, and it is indeed the lack of a checksum at the last 4 bytes or an incorrect checksum which cause the AFE to not be able to open the file with error: "Unable to load the file "D:\........" - it may have been corrupted". However, when you use the frw file to put it in your mixmaster sequence, RCT3 really doesnt care if the file has no checksum, or has it with an incorrect value or not.

  • Wow! 8| Just 6 seconds of video, but very impressive.


    Regarding checksums (CRC, Hash):

    There are OVL versions 1, 4 and 5. All CSO, CTR and CFR are OVL version 1 for basic attractions and ride features. The OVL versions 4 and 5 (from Soaked and Wild) have a checksum included, for which the algorithm is not known. Therefore I cannot write advanced OVLs from scratch. The relevant "long int" is included in SymbolStruct2 and SymbolRefStruct2 (Importer17 code).


    A similar checksum is protecting the main.ovl (included in the .chk file). If this algorithm would be known, an "Undo" button and other helpful additions could be programmed into RCT3. It seems that such a checksum (probably using the same algorithm) is included in .frw files, too. Do you know if there are different .frw versions (like 1, 4 or 5)?


    As far as I remember, Belgabor figured out about half of the Hash (CRC) algorithm. And JoeyWP did some investigations, too. But both never published their knowledge. So my conclusion is that you shouldn't waste your time to figure out the checksum.


    I am looking forward to watch your Efteling's fountain show, when you finally created it. :)

  • I just read on http://belgabor.vodhin.org/format/ that .frw structs have different versions v1, v2 and v3 (instead of 1, 4 or 5).


    And I found on my hard disc an old post from JoeyWP. Probably this helps you further.


    Hash algorithm:

    The hash algorithm is the so-called djb2 one, applied after converting the name to lower case.

    Code (provided str is already converted to lower case):


    unsigned long hash(unsigned char *str) {

    unsigned long hash = 5381;

    int c;

    while (c = *str++)

    hash = ((hash << 5) + hash) + c;

    return hash;

    }

  • FRW files indeed are in the versions v1, v2 and v3, where v1 has no header and v2 and v3 do have. I havent encountered any v2 frw files so far because i only have frw files from or RCT3 vanilla (v1 files), or RCT3 with soaked+wild (v3 files).
    Besides that v3 has the header (with extension) and v1 doesnt have a header, there are some more subtile differences. v1 files don't have the crc at the end as well, where v3 files do have the crc. Furthermore does the class-declaration in v1 frw files include all possible class declarations you could expect in any dat-type file (147 in total), where the v3 files only include the 3 class-declarations which are present in the file (ParticleBaseDesc, ParticleEmitterDesc & ParticleDesc for functional frw files, or only ParticleBaseDesc when you only have a base-emitter in your frw file).


    On the hash algorithm you posted, absolutely no clue how i should read that script to understand what it does :D. My experience and knowledge on programming is mainly limited to that from an engineering perspective using mathematics on physics. When it comes to actual computer science I'm mostly limited to understanding at least that everything is stored in bits, bytes .
    And indeed as you said, i will leave the checksum for what it is anyway. I got everything i need working and all work that is left for me to do is to write scripts which allow me to adjust color of other effects as well than just those of LED-lamps, but now I got all the tools I need to be able to do so.



    And to give you a better view of what I'm doing atm: Now I'm working on some more advanced patterns in the LEDs with the colors. In the video below you see a piece of around 62 seconds. everything one LED does within these 62 seconds is now put into one frw file for that specific LED, resulting in around 400 rows of code to generate 196 different frw files for the LEDs for a field of 260 LEDs (some LEDs use the same frw file):

  • I just uploaded several of my old files as kind of a mirror of my old Wonderplein.nl website here:
    https://rct3.craftventure.net/


    I think that for Markus the RCT3 archive dumper may be useful (not sure if I ever sent it to you by mail?):
    https://rct3.craftventure.net/…ArchiveDumper2_public.zip


    It runs over the file and explains every byte of it. That mirror also contains the Park CleanUp sources (they're crap though, I never did serious programming before that and I started with CPP) and some other sourcefiles. I don't know if everything still compiles, but there may be some usefull files over there.