Full access to prototype info at runtime

[sparr]

I've seen a number of requests go by on this forum for individual pieces of prototype information to be exposed via the API. It seems silly to do this inconsistently. How about just exposing every piece of prototype info through LuaPrototype?

[sparr]

Or, maybe, leaving `data` in-scope for reading at runtime, as a backup option for things not in LuaPrototype?

[ArderBlackard]

This has been answered already. The developers state that it would take a lot of time to expose the data.raw prototypes to the game-time code properly. So it's is unlikely to be done in the near future.

[Rseding91]

I've seen a number of requests go by on this forum for individual pieces of prototype information to be exposed via the API. It seems silly to do this inconsistently. How about just exposing every piece of prototype info through LuaPrototype?

The "data.raw" data doesn't represent the final state of prototype data as we tweak and process it after all mods are done adding information to it so just exposing the entire table would be harmful at best.

Was there something you wanted to read that isn't currently available runtime?

[sparr]

The "data.raw" data doesn't represent the final state of prototype data as we tweak and process it after all mods are done adding information to it so just exposing the entire table would be harmful at best.

Good reason for not exposing data.raw at runtime. Not an answer to why not everything in the final prototypes isn't exposed.

Was there something you wanted to read that isn't currently available runtime?

I've made a few posts to this forum with specific requests. So have other people. It just seems silly to keep making requests and exposing one thing at a time.

[Rseding91]

The "data.raw" data doesn't represent the final state of prototype data as we tweak and process it after all mods are done adding information to it so just exposing the entire table would be harmful at best.

Good reason for not exposing data.raw at runtime. Not an answer to why not everything in the final prototypes isn't exposed.

Was there something you wanted to read that isn't currently available runtime?

I've made a few posts to this forum with specific requests. So have other people. It just seems silly to keep making requests and exposing one thing at a time.

It's mostly about time. It takes time to add in the properties to be read and write tests to ensure they stay correct. There's a *lot* of stuff that can be accessed from the prototypes from basic things like entity max health to sprite animation speeds for a given entity at a given rotation.

I've added a large amount of things I saw missing for 0.13 but there's always more. Knowing what we should spend time on helps because just starting at the top and working down would take roughly a months time for one developer when 90% of it would probably never be used.

[orzelek]

I would say that problem is different - requests for this kind of access usually get forgotten.
At some point long time ago I tried to calculate %-age of oil field and found out I can't in runtime - I need minimum and normal values from prototype. Currently there is only minimum and request for normal was misunderstood and then lost after I posted clarification. It's also not present in 0.13.

[Adil]

I realy fail to see how dumping a copy of data.raw with readonly access can be hurtful.
It doesn't magically evaporate after the last data.lua is performed, does it

[sparr]

It's mostly about time. It takes time to add in the properties to be read and write tests to ensure they stay correct.

This part confuses and bothers me. If you're going to do it for everything then you don't do it by implementing them one at a time. You do it with decorators or preprocessor macros or similar, so that every current and future thing is automatically exposed.

Don't write a handler for belt speed and a separate handler for wire connection distance. Those are both float members of a struct somewhere. Write a single handler that exposes a C float as a lua float, and decorate *every* float member with that handler.

I don't know all the internals of Factorio, but I've been involved in this sort of API exposure in a few projects before, and writing new code for every variable you want to expose is never the right way.

[Rseding91]

I realy fail to see how dumping a copy of data.raw with readonly access can be hurtful.
It doesn't magically evaporate after the last data.lua is performed, does it

Yes, it does.

[Rseding91]

It's mostly about time. It takes time to add in the properties to be read and write tests to ensure they stay correct.

This part confuses and bothers me. If you're going to do it for everything then you don't do it by implementing them one at a time. You do it with decorators or preprocessor macros or similar, so that every current and future thing is automatically exposed.

Don't write a handler for belt speed and a separate handler for wire connection distance. Those are both float members of a struct somewhere. Write a single handler that exposes a C float as a lua float, and decorate *every* float member with that handler.

I don't know all the internals of Factorio, but I've been involved in this sort of API exposure in a few projects before, and writing new code for every variable you want to expose is never the right way.

You speak like you think each prototype is a single struct with all of the properties available right there in it. They're nothing like that. They're inherited/multi-inherited classes with other classes nested in them to any given level as properties. Some of them have other prototypes in them (triggers) and some are wrappers for things like sprites that point to external libraries.

[silverkitty23]

In my 24 of professional development experience, I have identified the following rule: "someone else's code is always easy to modify, even if they say it isn't."

[Adil]

You speak like you think each prototype is a single struct with all of the properties available right there in it. They're nothing like that. They're inherited/multi-inherited classes with other classes nested in them to any given level as properties. Some of them have other prototypes in them (triggers) and some are wrappers for things like sprites that point to external libraries.

Last time I've checked, data.raw was a plain lua table of plain tables, strings and numbers
Of the top of my head I can't name any property that when accessed through game interface has the different value from the one specified in data.raw.
If gremlins steal that one, could we have its deepcopy placed somewhere in gamescript, pretty please?

[sparr]

In my 24 of professional development experience, I have identified the following rule: "someone else's code is always easy to modify, even if they say it isn't."

I didn't say it would be easy to modify. I said it would be easier to modify to expose everything than to keep implementing one thing at a time.

[silverkitty23]

except I think Rseding91 was saying there isn't even an obvious definition of "everything" since not all the points are simple variables on simple objects. What does "expose everything" mean if a bunch of the things people ask for one at a time happen to require new code to make function calls to get the right value?

[sparr]

The function calls are already written to put the values there. We (mod writers, including whoever writes the base "mod" at Wube) give them data.raw, and that information ends up in all these crazy structs and classes and functions. All the information for putting the data in those places already exists in the mod loading code. Reverse it.

[sparr]

I wonder if there are enough NDAs I could sign that they would give me the code and let me try to throw together a reverse-engineer of the prototype loading process, as a proof of concept.

[Rseding91]

I wonder if there are enough NDAs I could sign that they would give me the code and let me try to throw together a reverse-engineer of the prototype loading process, as a proof of concept.

It's a possibility. That's how I originally started with Wube

[Rahjital]

Would it be possible to instead allow the data.lua and control.lua phases of mods communicate somehow? That way we could do this on our own without dev support, at our own peril.

[Rseding91]

The function calls are already written to put the values there. We (mod writers, including whoever writes the base "mod" at Wube) give them data.raw, and that information ends up in all these crazy structs and classes and functions. All the information for putting the data in those places already exists in the mod loading code. Reverse it.

That's what takes the time. Every single thing that we create from the data.raw table we have to make the opposite method to convert it back to the lua format. Then write a test that it's actually correct.