Jump to content
Official BF Editor Forums
Frankelstner

Bf4 Sbtoc Dumper

Recommended Posts

Grab the latest version of the script here: https://www.dropbox.com/s/rhu9gjxs9087vn7/bf4dumper.zip

Description of the LZ77 compression: http://pastebin.com/u2kntxSV

Description of the changes to the sbtoc: http://pastebin.com/TftZEU9q

Documentation of what I did to figure out the compression in case you are interested: http://pastebin.com/rGpBFwAV

Documentation of what I did to figure out the sbtoc changes: http://pastebin.com/0bZebD8S

Update 8.1.2014: Added support for the noncas archives (both unpatched and patched). I've rewritten most of the scripts along the way.

Update 17.1.2014: Fixed handling of noncas delta payload instructions: 1) Deltas without implicit base-reading section at the end are now handled correctly. 2) Fixed the sanity check for instruction type 3.

Update 1.2.2014: Noncas deltas ending with instruction type 4 are now handled correctly.

Edited by Frankelstner

Share this post


Link to post
Share on other sites

good work man, gonna try it out when i get home, just a question, will the audio decoder work on the files this dumper makes or do we have to wait a bit longer for you to make the decoder work with this?

Nope don't think so, on the beta files I keep getting this error.

Traceback (most recent call last):
 File "C:\Users\Durandal-217\Desktop\fb2audio\fb2decoder.py", line 624, in <module>
   decodeAudio()
 File "C:\Users\Durandal-217\Desktop\fb2audio\fb2decoder.py", line 65, in decodeAudio
   dbx=Dbx(f,unpack)
 File "C:\Users\Durandal-217\Desktop\fb2audio\fb2decoder.py", line 254, in __init__
   self.fieldDescriptors=[FieldDescriptor(self.unpack("IHHII",f.read(16)), self.keywordDict) for i in xrange(self.header.numField)]
 File "C:\Users\Durandal-217\Desktop\fb2audio\fb2decoder.py", line 109, in __init__
   self.name            = keywordDict[varList[0]]
KeyError: 1634562671
>>> 

I can't confirm if this happens with retail files.

Share this post


Link to post
Share on other sites

The decoder uses an ebx parser to grab the filenames. Obviously the script cannot magically know about the new ebx format. Will try to adjust it now, which should take just an hour or two if no other changes were made to the actual audio chunks. Though as you may imagine, I'm not sure what the chances are that the chunks were not modified as well.

Share this post


Link to post
Share on other sites

Strange unlogical error to me with the dumper...

I have these paths:

##Adjust paths here. The script doesn't overwrite existing files so set tocRoot to the patched files first,
##then run the script again with the unpatched ones to get all files at their most recent version.

bf4Directory=r"F:\Spel\Battlefield 4"
outputfolder="H:\Ljudeffekter\BF3 audio\bf4 dump"



catName=bf4Directory+"F:\Spel\Battlefield 4\Data\cas.cat" #use "" or r"" if you have no cat; doing so will make the script ignore patchedCatName too
patchedCatName=bf4Directory+"F:\Spel\Battlefield 4\Update\Patch\Data\cas.cat" #used only when tocRoot contains "Update"

tocRoot=bf4Directory+"F:\Spel\Battlefield 4\Update\Patch\Data\Win32"
##tocRoot=bf4Directory+"\\"+r"Data\Win32"

Get error message, telling me:

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 

Traceback (most recent call last):
 File "H:\Ljudeffekter\BF3 audio\dumper.py", line 401, in <module>
   cat=Cat(catName)
 File "H:\Ljudeffekter\BF3 audio\dumper.py", line 146, in __init__
   cat2=open(catname,"rb")
IOError: [Errno 22] invalid mode ('rb') or filename: 'F:\\Spel\\Battlefield 4F:\\Spel\\Battlefield 4\\Data\\cas.cat'
>>> 

Share this post


Link to post
Share on other sites

Yeah, well Im done now.. :D/> Didnt take 14 hours though.. Maybe 3.. :)/>

That's strange, my computer isn't all that old or slow, it dumped BF3's files in about an hour or more, but for BF4 it took 14 hours. I know frank said the script was slow, but for me that was just too slow.

Share this post


Link to post
Share on other sites

That's strange, my computer isn't all that old or slow, it dumped BF3's files in about an hour or more, but for BF4 it took 14 hours. I know frank said the script was slow, but for me that was just too slow.

I think anders dumped the beta files, not the full game. 14 hours seems like a good estimate.

I'm trying to improve the decompression (which is the bottleneck). With a small sample of files, the version here takes about 85 seconds. I've cut that down to about 25. I suppose when written in C I might get this down to 15 so most time is spent reading and writing the files and not decompressing.

Share this post


Link to post
Share on other sites

I dumped the full version game...

When looking over the EBX files etc. I discover LOTS of sounds etc.

So first I did as you recommended to dump the patched files first like this:

##Adjust paths here. The script doesn't overwrite existing files so set tocRoot to the patched files first,
##then run the script again with the unpatched ones to get all files at their most recent version.

bf4Directory=r"F:\Spel\Battlefield 4"
outputfolder="H:\Ljudeffekter\BF3 audio\dump5"



catName=bf4Directory+"\\"+r"Data\cas.cat" #use "" or r"" if you have no cat; doing so will make the script ignore patchedCatName too
patchedCatName=bf4Directory+"\\"+r"Update\Patch\Data\cas.cat" #used only when tocRoot contains "Update"

tocRoot=bf4Directory+"\\"+r"Update\Patch\Data\Win32"
##tocRoot=bf4Directory+"\\"+r"Data\Win32"

Then I dumped the non patched files like this:

##Adjust paths here. The script doesn't overwrite existing files so set tocRoot to the patched files first,
##then run the script again with the unpatched ones to get all files at their most recent version.

bf4Directory=r"F:\Spel\Battlefield 4"
outputfolder="H:\Ljudeffekter\BF3 audio\dump5"



catName=bf4Directory+"\\"+r"Data\cas.cat" #use "" or r"" if you have no cat; doing so will make the script ignore patchedCatName too
patchedCatName=bf4Directory+"\\"+r"Update\Patch\Data\cas.cat" #used only when tocRoot contains "Update"

tocRoot=bf4Directory+"\\"+r"Data\Win32"
##tocRoot=bf4Directory+"\\"+r"Data\Win32"

Had an error with the non patched files and with the script like this aswell ^^.

But when using the decoder, there are sounds that are cut offed and the sounds are FAR from beeing decoded. Like 20% are decoded...

I had more decoded sounds when I decoded the beta... So something when really wrong for me. :(/>/>

Edited by -=anders=-

Share this post


Link to post
Share on other sites

Nice! I was wondering where people were getting the ingame data.

Out of curiosity, is the compression (reverse) process trivial busywork or is it different?

Share this post


Link to post
Share on other sites

Compressing is a bit more complicated as it requires looking into the past of the file and finding matches that can be replaced. It's not that hard though.

However, it is not necessary to actually compress the files again. Each block specifies at the beginning whether it is compressed or uncompressed. Just using uncompressed data throughout the file with a little overhead (8 bytes for every 0x10000 bytes payload) is the most viable option. In fact I've done that when adapting the file tweaker to MOHW (which uses a similar format but with zlib). That one is somewhere on mordor and still working I believe (ugh, weapon damage mods; thankfully nobody cares).

Share this post


Link to post
Share on other sites

So this is what happened this time... Nothing I recognized and it took like 1 minute for both patched and original files:

>>> 
CommonChunks0.toc
CommonChunks1.toc
FrontEndChunks.toc
Globals.toc
MpCharacter.toc
MpChunks0.toc
MpChunks1.toc
MpVehicles.toc
Persistence.toc
PrologueChunks.toc
SpChunks0.toc
SpChunks1.toc
UI.toc
WeaponChunks.toc
WeaponsAndAttachments.toc
FrontEnd.toc
MP_Abandoned.toc
MP_Damage.toc
MP_Flooded.toc
MP_Journey.toc
MP_Naval.toc
MP_Playground.toc
MP_Prison.toc
MP_Resort.toc
MP_Siege.toc
MP_TheDish.toc
MP_Tremors.toc
SP_Airfield.toc
SP_Dam.toc
SP_Naval.toc
SP_Prison.toc
SP_Prologue.toc
SP_Shanghai.toc
SP_Suez.toc
en.toc
en_CommonVO.toc
en_MpVO.toc
en_SpVO.toc
>>> 

Only toc files? I dont get it. :)

Share this post


Link to post
Share on other sites

But getting errors:

Traceback (most recent call last):

File "F:\Development\Ninja_Ripper\Battlefield 4 dumper\dumper.py", line 13, in <module>

LZ77 = cdll.LoadLibrary("LZ77")

File "C:\Python27\lib\ctypes\__init__.py", line 431, in LoadLibrary

return self._dlltype(name)

File "C:\Python27\lib\ctypes\__init__.py", line 353, in __init__

self._handle = _dlopen(self._name, mode)

WindowsError: [Error 193]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×