Jump to content
Official BF Editor Forums
Sign in to follow this  
PiratePlunder

Dice Release Some 'new' Modding Info For Bf3

Recommended Posts

Yesterday Mikael Kalms (DICE) released some newish information about why it's not possible to release mod tools for BC2 (Frostbite 1.5) and raised the spectre of modding on BF3 (Frostbite 2.0). Anyway, here's the post copied across from the EAUK forums I'll let you decide what you think, but my thoughts are at the bottom. It's very nice to have a seemingly honest account of the situation from DICE directly.

Zh1nt0 and you folks have asked about it' date=' so here's a piece on the modtools situation for BC2 PC.

Frostbite 1.5 consists of these components:

* The game runtime

* The editor runtime

* The content processing runtime (aka "the pipeline")

* and some plugins for Maya

The game runtime is distributed outside of EA, but the editor + pipeline + Maya plugins are not.

So let's take a look at some things that would need to be solved before we'd be ready to distribute the editor + pipeline.

[b']Pipeline operation[/b]

Let's say that you tell the pipeline to build level MP_003.

MP_003 is represented by an XML file, which references a bunch of other files. These in turn reference other files. If you follow this graph of references, you will find the level layout, heightmap, characters, weapons, vehicles, and all the content that you can see in-game. (The in-game HUD and related stuff might also be in the graph.)

When the pipeline is about to build MP_003, it will first perform a consistency check on all content, and yell if any file that is referenced by any other is not present.

If all files are present, the pipeline will attempt to convert all files referenced by MP_003. It uses the file system journal to determine which files have changed on-disk. Also, and any files that have already been converted have info on which files depend on it (so it has info like: "if file X changes, then files Y,Z,W will also need to be rebuilt").

Building all content for BC2 from scratch takes something like 48-72 hours on a normal workstation. Half that time is spent building common content (such as character animations), half builds level-specific content.

In addition, there's a caching mechanism: if the pipeline wants to build a specific bit of content, it will first check if the pre-built content is already available on a cache server and take the result directly from the cache server instead. The pipeline can also populate the cache if it builds something new.

Pipeline issues

So how does this work in practice? It's not ideal, but it's good enough for us to ship games on it.

The pipeline is a bit overzealous with regards to rebuilding assets - sometimes it rebuilds stuff that it shouldn't need to.

The pipeline will normally crash about 2-3 times during a full rebuild.

You need to have Maya 8.5 (32-bit version) installed in order to convert any meshes.

Any content in the cache expires after 3 weeks. After 3 weeks have passed, that content will need to be rebuilt and re-uploaded by a machine running the pipeline. The effect that this has on day-to-day development is minimized by having one or two machines dedicated to running the pipeline every time any content change is done. By running the pipeline, those machines will populate the cache, thereby speeding up the build process for everyone else. (The output form those content build steps is discarded.)

In short: the pipeline + cache setup works better the more people are using it simultaneously.

If there are content errors, you need to know a lot about the internals of the game engine to figure out what's wrong.

Finally, in its current form, the pipeline + editor expects some specific IT infrastructure in place (most notably the cache server and a Perforce server).

If it's not there then the pipeline + editor will behave strangely.

The first time I tried, it took me about one week to get the full editor + pipeline setup to work properly outside of the DICE office. And that was when I had the option to call any of the other developers to ask for help.

... does this sound bad to you?

Truth be told, this is approximately where the industry average is at for game studios' internal game engines. One of FB 1.5's weaknesses is specifically that its content processing is flaky, and the flakiness gets more problematic as the amount of content goes up. FB 2.0 is much improved in this regard, but FB 1.5 is what we're using for BC2 and that's what relevant in the current discussion (or monologue if you prefer).

Content

Both the pipeline and the editor takes in all content in its raw, original form. Anyone who is to build any content needs the full 80GB of raw data on their machine. We are not comfortable giving out all our animations, meshes etc in raw form.

We are comfortable giving out the processed data - after all, that's what on the game disc - but that data does not plug into the editor/pipeline at all.

Licenses

The game, editor and pipeline all use commercial middleware. It is developed by Havok and several other companies.

The licensing agreement for the middleware allows us to use that code in specific products, on specific platforms.

If we want to release editor + pipeline, we need to license the middleware specifically for this. How much would that be? Perhaps $1M-$3M. I'm guessing wildly here.

Stripping out that middleware would seriously hamper the functionality especially of the pipeline. We use Havok Physics, for instance. Without Havok Physics, the pipeline wouldn't be able to convert any of the physics meshes. We also use Granny. Without Granny, the pipeline will not be able to convert any of the character animations. Etc.

Re-implementing the necessary functionality of the middleware ourselves ("let's make our own physics engine / let's plug in an open-source physics engine") would take literally man-years. Licensing is cheaper in pure $ cost and faster (it works now instead of by 2012).

The pipeline also uses some code that is under GPL. Given that we do not want to release the full source code for the editor + pipeline, we would need to replace the GPLed code with other implementations.

The GPLed code is less of a problem than the proprietary middleware.

Editor

The editor itself is reasonably stable and well-behaving. It is far from obvious how to set up the game logic for a level, but that is easily covered by releasing some example levels which contain the logic setup for the common gamemodes.

Test-running levels

First the level needs to be successfully processed by the pipeline. Then you'd want to be able to test it locally. That involves having a listen server around. We don't have a listen server neatly packaged. There's probably a piracy angle here too but I'm not going to discuss that.

Distribution of levels

Getting levels onto the RSPs server machines would likely not be any problem. However there's need for checksumming levels, so that game clients can know whether or not they have the correct version of level X on their machines. There's a whole bunch of other things (mainly UI-related) which will need cleaning up as well. Not difficult to do, just takes time and I'm listing it for the sake of completeness.

Also, there are some complications wrt when we release patches that affect the base game's content. Whenever we release a patch, all existing levels will need to be rebuilt with a new set of original data. This is because some level-common data is stored inside of the level archives. I'm not sure at the time of writing, but that probably means that the only manageable way for us would be to invalidate any user-made levels when we release a patch of that form.

Then creators of any user-generated levels would be required to run their levels again through the pipeline with the new base content supplied.

So how about just a map editor?

If it doesn't plug into the ecosystem above, then getting it to work involves some serious wrangling. Either it is a light-weight replacement for our existing editor - in which case all the challenges with the pipeline still remain - or it is a separate mode (think Forge for Halo). Developing an extra mod-layer that is sandwiched into the game would easily take 6-12 months.

Synergy effects between FB 1.5 and FB 2.0

So let's say that we would go through the procedure of making mod tools for FB 1.5. How much of that work would be reusable for FB 2.0?

I don't have any firm figures, but the differences between FB 1.5 and FB 2.0 are pretty large by now. Given this and the fact that a fair bit of the FB 1.5-specific problems (where the devil often is in the details) don't apply to FB 2.0, I'd guess that less than half of the work would port over to FB 2.0.

Conclusion

In conclusion, my recommendation to the rest of DICE is not to develop mod tools for BC2 PC. There are too many hurdles to overcome. That energy is better spent elsewhere, be that on BC2 or other titles.

So, where does that leave modding for BF3. In my opinion pretty much dead.

There's a lot of talk about the pipeline and the time it takes to build the content as a couple of days and this is where you can certainly see that improvements in FB2.0 might make a difference. Trouble is for mod teams this is never going to be a problem. BF2 mappers are already used to lightmapping a single level taking the best part of a day and I'm sure that most of the larger teams probably had at least one member of staff that had a spare computer set aside for a particular modding task, be it lightmapping or whatever - I know we did. I also know that we would rarely have gone more than three weeks without creating a build as well.

The need for maya isn't really a show stopper - BF2 mod teams pretty much needed either Maya or Max (I know some teams struggled on with Gmax) and although ensuring that whoever runs the build had maya would require some consideration I'm guessing it would take mod teams about 5 minutes to organise that. From what I can see Perforce is free for small teams, and I know from our experience companies were happy to provide us a server for an SVN for free. So don't see a problem there.

Finally for the pipeline, if it took a DICE dev a week to get it set up and working at home, I'm guessing some of the guys here would have it set up within 24 hours and in that time would also have written a step by step tut. Within a week we'ld have some software to simplify some of the setup tasks.

Content and Licenses are what they are and I'm in no position to disagree with them. Except to say other teams have managed it. I believe crytek use Havok yet have an amazing map editor (and less than amazing mod tools). The figures used for licensing sound far fetched but you have to believe that EA/DICE are never going to pay good money to deliver mod tools - using resources/time perhaps but not $$$'s. The other issue is of course that modding would require dedicated servers, seemingly a DICE no no now.

So whilst overall that doesn't sound too bad - the pipeline issues are largely non issues, with the exception of DICE not wanting to hand over their raw files, and you would think that a way around the licensing issues could be found. The killer though is in another post later in the thread...

FB2.0 is better suited for modtools' date=' but it is not a shoe-in yet. I will not speculate on whether or not modtools will be released for BF3.[/quote']

If the team don't yet know if mod tools will be released for BF3 then they wont. The ability to have mods run needs to be built into the engine at the start otherwise its too late. Just prior to release and in the six months plus after release the team will be too busy trying to fix all the bugs that will inevitably be there. Six months after release they'll be heavy into the next title and the hotfixes for BF3 will make supplanting mod capabilities even harder.

Add to that I have doubts about how much really has changed between FB1.5 and FB2. The use of havoc or granny are almost certainly the same and whilst the pipeline may be cleaner (due to the problems they've had building and more specifically patching) the problems that really are problems - dedicated servers, release of raw assets, cost of licenses and time to develop external tools - will all remain. I can imagine it being possible to make the compiler use pre-compiled assets so that a level designer was possible but even that I think is unlikely due to having to test a map, no dedicated servers etc.

I just don't see modding for BF3 happening, although I'ld love to be proved wrong, the ability to mod, to run mods is a requirement that touches so many aspects of the game. The UI, the file structure, the compiler, the servers. I'm just left with the increasing feeling that as horrible as modding BF2 was at times and as annoying as it could be when you came up against something hardcoded that never should have been, we were so lucky to have it, its file structure and incredibly simple pipeline where everything could be done from Max and notepad, and that ultimately things will never be so good again.

Edited by PiratePlunder

Share this post


Link to post
Share on other sites

thanx for the bad news plunder...lol....your last paragraph was well said,we have been very lucky indeed.

.

it blows me away they moved so far away from the BF2 pipeline and file structure,its awesome!,they could have easily improved the refractor engine to build bfbc and bf3...some new shaders,some physics improvments and improvments to singleplayer missions.

no mod tools is a loss for us and a loss for DICE,gamers will get bored and move on to the next big thing.

Share this post


Link to post
Share on other sites

Its a loss for us, but not for dice, as dice is ea owned, that next big thing will no doubt have ea on the cover too, so long as theres always more ea games on the shelf than anyother they not care about long life games anymore, all just quick bucks console games, that people trade in when bored of for the newest game... which is no doubt an ea game :P

Share this post


Link to post
Share on other sites

I'm still of the opinion that if BF3 doesn't have mod support, I'm passing it by. I know that MANY people say this (recently with MW2, saying one thing and buying it on release day), however I'm not a person with poor impulse. If BF3 arrives without mod support and no dedicated servers, all future games from DICE should be considered fruit of the poisoned tree and treated just as harshly as MW2.

Its time that DICE just move on from making PC games and just create shovel-ware for MS and Sony. I won't miss them and move to another engine with no tears shed.

Edited by Mach10

Share this post


Link to post
Share on other sites

thanx for the bad news plunder...lol....your last paragraph was well said,we have been very lucky indeed.

.

it blows me away they moved so far away from the BF2 pipeline and file structure,its awesome!,they could have easily improved the refractor engine to build bfbc and bf3...some new shaders,some physics improvments and improvments to singleplayer missions.

no mod tools is a loss for us and a loss for DICE,gamers will get bored and move on to the next big thing.

Beex - lol np. Sadly the reason for Frostbite i expect is Consoles. The reason for having to pack each map seperately the way they do - consoles. I guess they couldn't handle opening lots of archives with unused content.

Share this post


Link to post
Share on other sites
Sadly the reason for Frostbite i expect is Consoles
Indeed , and i guess its the main reason for the processing 'pipeline' . It can probably be configured to target those different platforms ( comparable to building a linux kernel ) and with all the licensed middleware its an easy way for a developer team with a central fileserver structure to built their stuff.

Its probably the end of modding for DICE products - but i feel that Refractor-BF2/2142 hasn't reached its EOL :D There's gamers out there for it, it has a relatively robust structure and its easy to mod.

Share this post


Link to post
Share on other sites

honestly, iam not suprised about that at all.

i mean ok, bf2 was and is still awsome to mod but after bc2 and the strange actions going on with bfh, bf1943 and all the patches n' stuff i rly dont give a f**k.

if this is their way, they wanna produce games and do things like they did it the last 1,5 years - ye iam fine with it :P

life goes on, time pass' by - iam sure every single dude inside of this forums will find his own way, without dice.

i knew it already that bf3 will be more a bc3, but iam sure they know as we know it too, a bf3 will bring more attention and also a lot more of money. and because of money makes the world go round anyway, why not trade a c into an f and all the lemmings will follow :lol:

Share this post


Link to post
Share on other sites

Well, if we were more convinced that the engine would actually allow you to make mods without ruining your copy of the game I think we'd be more inclined to puzzle through all those formats and make our own editor, in reverse. I mean, it's a finite combination of bytes - so it ought to be POSSIBLE to make our own tools, but considering how tightly locked-in it is, I can't see that happening.

Seems that licensing thing is the real killer in all this, and probably explains the proprietary formats too (companies will often have it as a requirement that the end user mustn't be able to use the files in the game).

Disappointing but not surprising news, oh well. But perhaps we can pursue that avenue of asking DICE for better access to the Refractor 2 engine, because we're familiar with it and it can be made more powerful. A lot of mods have shown that the coding constraints placed by DICE can be sidestepped to achieve some surprising results, and high poly models, high resolution textures, and good shaders will go a long way as well. BF2 already has a solid network system in place, but physics and shadows are really in need of love, and some more persistent barriers between the realms of possible and impossible on the Refractor 2 engine could do with some demolition.

Edited by UberWazuSoldier

Share this post


Link to post
Share on other sites
But perhaps we can pursue that avenue of asking DICE for better access to the Refractor 2 engine, because we're familiar with it and it can be made more powerful.

Yep that was my thinking also and have for the last few days been speaking to some DICE guys to that end and they were kind of warm to the idea. My gut feeling is that there is still at heart a love/respect of PC gaming at DICE (at least those that haven't been drafted in post BF2) due to their history right back to the pinball trilogy but ultimately an understanding that commercially consoles are the future. Unfortunately continuing sales of BF2, BF2142 and obviously the continuation of Battlefield Heroes means that at the moment the answer has been a strong no to releasing refractor under some kind of GPL or even licensed along the lines of UDK.

However, like I say there was some degree of warmth to the idea - it wasn't just ignored out of hand which would have been an easy thing to do. With that in mind and with the upcoming(ish) release of BF3 (and what that will do for sales of BF2,BF2142 and BFH) and the bad publicity that the eventual announcement that there will be no BF3 modding capabilities (I'm as good as certain of it) will have then perhaps this is an avenue worth pursuing. If we can get a large part of the BF2 modding community together and behaving in a constructive and grown up manor then it might be that we can get enough weight behind the idea to persuade DICE of the benefits of such a move.

A few of the key issues I would see are:

1) How to ensure that the release of Refractor wouldn't damage ongoing sales/revenue from their existing games. BF2, BF2142, BFH.

2) How to ensure that the release of Refracotr wouldn't damage ongoing sales/revenue form DICE's future games. BF3 BC whatever.

3) How to maximise the good publicity for DICE of releasing Refractor and the awareness that refractor is DICE.

There may be one big stumbling block that stops DICE releasing refractor. What if the potential of refractor as a PC based FPS engine is greater than that of Frostbite? FB has clearly been developed as multiplatform with if anything the main emphasis on consoles. Perhaps we'll see different with BF3, but my gut feeling is that the map sizes, file structure etc is all down to the various limitations of a console. It is therefore possible that DICE would be concerned that with a bit of luck and some determined mod teams that FB could actually be outdone by DICE own outdated engine. That would be pretty damming. Of course, this is purely speculation.

Edited by PiratePlunder

Share this post


Link to post
Share on other sites

"id Software has released the source code to Return To Castle Wolfenstein (single player and multiplayer), along with Wolfenstein – Enemy Territory, under the GPL. The linked article notes that "these only include the game source code, not the graphics. You need the full games for those."

Perhaps something along these lines? Given the above, the security blanket for the developer will be the state of the graphics until the current ReFractor games reach end-of-life. Perhaps add-in an option that DICE is given first crack at purchasing any new IP created from the source release to sweeten the deal? A glaring example of the revenue created from acquired modding IP is clearly Valve. Team Fortress, Garry's Mod, Portal... those titles are creating tens of millions of dollars for them.

Edited by Mach10

Share this post


Link to post
Share on other sites

Well source is a great language since it uses the "mod files" directly instead of loading them. But source can't offer large maps, vehicles, kits etc. This all because the source engine doesn't use build-in (hardcoded) stuff, like team values, spawn points, and a lot of other stuff.

The hardest thing in building a moddable game is linking the modder with the c++ code. It almost impossible to store game structures (like an entire vehicle) as a file on the harddrive. Also, to access the file you need to load the raw data into memory, usually this is impossible since it can not cast the data as a game structure type.

So what did EA do for bf2 and bf2142? They made a huge wrapper around their game, all files are loaded into structures separately, no raw data involved. Downside? Loading is extremely slow. Loading a text file line by line takes approx. 500 ms for a regular file. Make this 20000 files and you got a nice 10000 sec load time, and this is excluding textures, meshes and preparing the hardware. For larger games this is just not an option, and you don't want to wait for 30 minutes only because the game is moddable, do you?

So EA stepped from a slow loading time and moddable to a fast loading time and not moddable. Raw data doesn't need text processing, you load it and it's there. You don't need to loop through all files because the data IS there, and all errors are already out. No debugging needed either, fastens the game up even more.

To be fair, I agree with DICE for changing to a steady, fast and reliable game. The problem is that today's game prices drop dramatically after a few months of release. Battlefield 2 is now less than 10 Euro, while I bet it was a lot more years ago. DICE/EA goes for the money and the fast game production, just to counter the falling game prices. :(

And they got some competition from the coming wave of Japanese video games. :D

Share this post


Link to post
Share on other sites

Hmm, PiratePlunder has some good points.

1) How to ensure that the release of Refractor wouldn't damage ongoing sales/revenue from their existing games. BF2, BF2142, BFH.

Well, if we had the modifications in the form of a patch for those respective games, then it would still require people to buy the games (and would boost sales). But I think we're all more thinking along the lines of a standalone game, which... makes that tricky indeed.

2) How to ensure that the release of Refractor wouldn't damage ongoing sales/revenue form DICE's future games. BF3 BC whatever.

Tough, but I think the people who are interested in indie-ish games aren't the sort who would buy Bad Company 2 and Battlefield 3.

3) How to maximise the good publicity for DICE of releasing Refractor and the awareness that refractor is DICE.

Spam DICE logos everywhere I guess?

I mean, in time I'm sure they would come around to releasing it if we asked... But how much time that will be seems to be the problem.

Edited by UberWazuSoldier

Share this post


Link to post
Share on other sites
Sign in to follow this  

×