Bf4 Sbtoc Dumper

[ssbeye]

There one problem )) Normal maps are looks like this:

But alpha inside looks fine:

Edited November 12, 2013 by ssbeye

[JackusCTB]

@ssbeye

How did you convert the textures from chunks?

[ssbeye]

@ssbeye

How did you convert the textures from chunks?

Chunks to DDS Converter .. tool for BF3 .. here in other post )))

[infinityloop]

Chunks to DDS Converter .. tool for BF3 .. here in other post )))

You mean that one http://www.bfeditor.org/forums/index.php?showtopic=15731&view=findpost&p=106657

or is there an other one too?

[ssbeye]

You mean that one http://www.bfeditor.org/forums/index.php?showtopic=15731&view=findpost&p=106657

or is there an other one too?

Yes )))

[elementofprgress]

New patch out, i'm getting errors now. Kinda feel like I broke something somewhere or doing something wrong on my end lol. Saying that in the toc files shit doesn't start with the \x82 even though I look in a hex editor and its starts with \x82. Anyone have any luck? Figured i'd ask before start looking threw stuff to see if anything in the scripts need to be changed.

Its not just me.

Traceback (most recent call last):

File "C:\Bin\bfscripts\dumper.py", line 582, in <module>

main()

File "C:\Bin\bfscripts\dumper.py", line 579, in main

dump(fname,outputfolder)

File "C:\Bin\bfscripts\dumper.py", line 230, in dump

toc=sbtoc.Superbundle(tocName)

File "C:\Bin\bfscripts\sbtoc.py", line 181, in __init__

self.entry=Entry(toc)

File "C:\Bin\bfscripts\sbtoc.py", line 90, in __init__

raise Exception("Entry does not start with \x82 or (rare) \x87 byte. Position: "+str(toc.tell()))

Exception: Entry does not start with ‚ or (rare) ‡ byte. Position: 1

>>>

Even though the error says what it says. the entries do seem to start with \x82. BUT now there is \x00\xD1\xCE\x03

Edited November 14, 2013 by elementofprgress

[Frankelstner]

Even though the error says what it says. the entries do seem to start with \x82. BUT now there is \x00\xD1\xCE\x03

Note that the entries in question are not the ones in the sb, but in the toc. I've changed the unXOR function in the sbtoc.py. The tocs are not encrypted anymore. In fact the game does not contain any encrypted files at all now.

[elementofprgress]

Note that the entries in question are not the ones in the sb, but in the toc. I've changed the unXOR function in the sbtoc.py. The tocs are not encrypted anymore. In fact the game does not contain any encrypted files at all now.

That is what I get for looking at this stuff half awake at 6am. The whole no more encryption thing threw me for a loop. While half awake it made me question what I was looking at lol.

Did you notice the XP1Weapons SB/Toc and XP1WeaponsChunks? I haven't looked at them yet(should probably really really study your notes first before I do lol) but seems like the scripts say they are the noncas bundles. Took a quick glance and they look a lot like the Xpack sbtocs. IDK I find it odd that they added anything to the data folder.

Anyways, like I said I only glanced at the files but thought I would let you know about them.

[Frankelstner]

The script now supports the xpack archives. Note that you must replace previous versions of the LZ77.dll as I had to make some extensive changes to it too.

Meh, I forgot to make tocRoot2 absolute so the unpatched files were not extracted. I've fixed that though and on the bright side, if you happen to have used the old version you can just run the new one and it will continue where the other one left off.

Edited January 8, 2014 by Frankelstner

[Durandal-217]

The script now supports the xpack archives. Note that you must replace previous versions of the LZ77.dll as I had to make some extensive changes to it too.

Meh, I forgot to make tocRoot2 absolute so the unpatched files were not extracted. I've fixed that though and on the bright side, if you happen to have used the old version you can just run the new one and it will continue where the other one left off.

No issues thank you so much for all the hard work you put into this.

[dainiuxxx]

Yes, thank you Frank you are the hero of this community! ;]

Edited January 10, 2014 by dainiuxxx

[wirrew]

Hi people of the bfeditor forums! This is my first post here so bare with me

I've tried to use this script for like a week now but I'm still getting errors. I've tried this on two different operating system setups (Win7/8) on the same PC, where each gives a unique error.

When running the script on my Win8 setup, I get this error after the dumping process is "finished":

Traceback (most recent call last):

File "C:\Python27\bf4dumper\bf4dumper.py", line 257, in <module>

if "tocRoot" in locals(): dumpRoot(tocRoot)

File "C:\Python27\bf4dumper\bf4dumper.py", line 248, in dumpRoot

dump(fname,targetDirectory)

File "C:\Python27\bf4dumper\bf4dumper.py", line 179, in dump

writePayload(entry, targetPath, sourcePath)

File "C:\Python27\bf4dumper\bf4dumper.py", line 224, in casPatchedPayload

casPayload(bundleEntry, targetPath, sourcePath) #if casPatchType is not 2, use the unpatched function.

File "C:\Python27\bf4dumper\bf4dumper.py", line 213, in casPayload

LZ77.decompress(catEntry.path,catEntry.offset,catEntry.size, bundleEntry.originalSize,targetPath)

WindowsError: [Error -529697949] Windows Error 0xE06D7363

I also only end up with about 22GB of dump files. I've heard it's supposed to be about 60GB.

When I run the script on my Win7 setup, I get this error before the process even starts:

Traceback (most recent call last):

File "F:\Battlefield Decoding\stuff for dump\bf4dumper.py", line 43, in <module>

LZ77 = cdll.LoadLibrary("LZ77")

File "C:\Python27\lib\ctypes\__init__.py", line 443, in LoadLibrary

return self._dlltype(name)

File "C:\Python27\lib\ctypes\__init__.py", line 365, in __init__

self._handle = _dlopen(self._name, mode)

WindowsError: [Error 126] Det går inte att hitta den angivna modulen

>>>

The last line (Swedish) says "Can't find the stated/defined module"

I hope that someone here can help me fix this so I'm able to complete the dumping process w/o any errors. Please ask if you need any more information.

Thanks in advance!

// wirrew

Edited May 3, 2014 by wirrew

[dainiuxxx]

make sure you have the right version of the python, and keep trying, it is not uncommon to run into issues with this.

here is an old vid I made a while back, might be of some help...

Edited May 3, 2014 by dainiuxxx

[wirrew]

 

make sure you have the right version of the python, and keep trying, it is not uncommon to run into issues with this.

here is an old vid I made a while back, might be of some help...

 

I do have the correct version of python (2.7.6) and I've tried everything that I could think of. Anyone else having any tips or ideas?

[DEVILMAN]

1) Download Python here: http://www.python.org/ftp/python/2.7.6/python-2.7.6.msi

2) Right click on the bf4dumper.py file, choose "Edit with IDLE".

3) Adjust the script if necessary. Save with Ctrl+S.

4) Run the script by pressing F5. A shell will open showing the names of extracted files. Note that the asterisks in the title of the shell indicate that the script is running.

5) The script is finished when the asterisks are gone and a new line (>>>) awaiting further input appears.

What about the other two py scripts?

They are imported by the dumper script. It's not necessary to modify them and they don't do really anything when you run them.

Where to place dlls/auxiliary py scripts?

Either next to the bf4dumper.py or in the Python folder, i.e. C:\Python27.

I tend to put the dlls in the Python folder and leave the scripts next to each other.

You dont need to Edit with IDLE,you can change the paths when the py file is set to open with notepad

[wirrew]

 

1) Download Python here: http://www.python.org/ftp/python/2.7.6/python-2.7.6.msi2) Right click on the bf4dumper.py file, choose "Edit with IDLE".3) Adjust the script if necessary. Save with Ctrl+S.4) Run the script by pressing F5. A shell will open showing the names of extracted files. Note that the asterisks in the title of the shell indicate that the script is running.5) The script is finished when the asterisks are gone and a new line (>>>) awaiting further input appears.What about the other two py scripts? They are imported by the dumper script. It's not necessary to modify them and they don't do really anything when you run them.Where to place dlls/auxiliary py scripts? Either next to the bf4dumper.py or in the Python folder, i.e. C:\Python27. I tend to put the dlls in the Python folder and leave the scripts next to each other.You dont need to Edit with IDLE,you can change the paths when the py file is set to open with notepad

 

Yeah, these were the instructions that came with the download. I followed them but this still hasn't changed anything.

[DEVILMAN]

@wirrew

you must enter the proper path for your game

patched and unpatched games,require a different path name

also patched and unpatched games will give a different output file size

[DEVILMAN]

I just dumped BF4

My data folder is 27.4 GB

My update folder is 603 MB

My extracted folder is 38.7 GB

My paths are

catName=r"C:\Battlefield 4\Data\cas.cat" #use "" or r"" if you have no cat; doing so will make the script ignore patchedCatName

patchedCatName=r"C:\Battlefield 4\Update\Patch\Data\cas.cat" #used only when tocRoot contains "Update"

tocRoot=r"C:\Battlefield 4\Data"

Dump time 25-30 minutes

[wirrew]

I just dumped BF4

My data folder is 27.4 GB

My update folder is 603 MB

My extracted folder is 38.7 GB

My paths are

catName=r"C:\Battlefield 4\Data\cas.cat" #use "" or r"" if you have no cat; doing so will make the script ignore patchedCatName

patchedCatName=r"C:\Battlefield 4\Update\Patch\Data\cas.cat" #used only when tocRoot contains "Update"

tocRoot=r"C:\Battlefield 4\Data"

Dump time 25-30 minutes

By looking at your folder sizes, I assume you don't have all the DLCs that are available. I do have all the DLCs (XP0,XP1,XP2) and my extraction folder as stated earlier is only about 22GB, yours is almost 39GB. This tells me that something is way off as I've stated in my very first post.

Also, here are my paths, if i try to change them to a manual path, I get error saying it could not find the path.

bf4Directory = r"G:\Program Files (x86)\Origin Games\Battlefield 4"

targetDirectory = r"F:\Battlefield Decoding\Battlefield 4 Dump"

tocRoot = r"Update" #patched and xpack files FIRST

tocRoot2 = r"Data" #unpatched vanilla files SECOND

catPath = r"Data\cas.cat"

updateCatPath = r"Update\Patch\Data\cas.cat"

Edited May 8, 2014 by wirrew

[DEVILMAN]

apparently,you need to run it twice,if you want to dump it all

38 GB,22 GB = 60 GB

Try overriding your catPath path,by changing the catName path,as I did

catName=r"G:\Program Files (x86)\Origin Games\Battlefield 4\Data\cas.cat" #use "" or r"" if you have no cat; doing so will make the script ignore patchedCatName

[wogoodes]

I'm starting work on converting this script for Dragon Age: Inquisition. I have no experience with Python, reverse engineering, or most asset formats, so this will probably not be pretty!

The current upshot is that DA:I actually uses zlib (which will probably anger frankelstner). zlibbed chunks are headed with 0x0270 rather than the LZ77-specific 0x0970 header. Everything else I've seen is the same.

While it would be best to have the script work for both zlib and LZ77, I'm currently not competent enough to do that, so for now, I'm simply going to allow the zlib header to be recognized, and then I will replace all uses of LZ77 with zlib.

First cut :

Replace casPayload in bf4dumper.py with the code below :

#for each bundle, the dump script selects one of these four functions
def casPayload(bundleEntry, targetPath, sourcePath):
    catEntry=cat[bundleEntry.sha1]
    print targetPath
    print catEntry.path
    print catEntry.offset
    print catEntry.offset+12
    print catEntry.size
    print catEntry.size-12
    print bundleEntry.originalSize
    print targetPath
    casFile=open(catEntry.path,"rb")
    casFile.seek(catEntry.offset+8)
    decompressor = zlib.decompressobj()
    targetFile = open(targetPath,"wb")
    compressedStream = casFile.read(catEntry.size-8)
    decompressedStream = decompressor.decompress(compressedStream)
    targetFile.write(decompressedStream)
    #LZ77.decompress(catEntry.path,catEntry.offset,catEntry.size,bundleEntry.originalSize,targetPath)

Replace seekLZ77Block in noncas.py with the code below :

def seekLZ77Block(f):
    decompressedSize, compressionType, compressedSize = unpack(">IHH",f.read(8))
    if compressionType in (0x70,0x71,0): f.seek(decompressedSize,1)
    elif compressionType==0x970:         f.seek(compressedSize,1) 
    elif compressionType==0x270:         f.seek(compressedSize,1) 
    else: print "Unknown compression type:",str(compressionType), "Offset:", f.tell(),bad
    return decompressedSize

Result : we get a partial dump! Only a few files, really. Still, progress. Edited November 30, 2014 by wogoodes

[MrDude]

Your efforts are very much appreciated. I replaced the bits like you said but:

layout.toc
frostbackmountains.toc
X:\HereticalGames\Dragon Age Inquisition\DA3EXT/bundles/res/da3/factions/generic/props/textures/prp_josephinebrush_s d9780f918f57db83 0b000000030000000000000000000000.itexture
X:\HereticalGames\Dragon Age Inquisition\DA3EXT/bundles/res/da3/factions/generic/props/textures/prp_josephinebrush_s d9780f918f57db83 0b000000030000000000000000000000.itexture
X:\HereticalGames\Dragon Age Inquisition\Data\cas_18.cas
1064183689
1064183701
93
81
128
X:\HereticalGames\Dragon Age Inquisition\DA3EXT/bundles/res/da3/factions/generic/props/textures/prp_josephinebrush_s d9780f918f57db83 0b000000030000000000000000000000.itexture

Traceback (most recent call last):
  File "X:\HereticalGames\Dragon Age Inquisition\bf4dumper\bf4dumper.py", line 272, in <module>
    if "tocRoot" in locals():  dumpRoot(tocRoot)
  File "X:\HereticalGames\Dragon Age Inquisition\bf4dumper\bf4dumper.py", line 263, in dumpRoot
    dump(fname,targetDirectory)
  File "X:\HereticalGames\Dragon Age Inquisition\bf4dumper\bf4dumper.py", line 179, in dump
    writePayload(entry, targetPath, sourcePath)
  File "X:\HereticalGames\Dragon Age Inquisition\bf4dumper\bf4dumper.py", line 239, in casPatchedPayload
    casPayload(bundleEntry, targetPath, sourcePath) #if casPatchType is not 2, use the unpatched function.
  File "X:\HereticalGames\Dragon Age Inquisition\bf4dumper\bf4dumper.py", line 223, in casPayload
    decompressor = zlib.decompressobj()
NameError: global name 'zlib' is not defined

Edited November 30, 2014 by MrDude

[wogoodes]

Your efforts are very much appreciated. I replaced the bits like you said but:

global name 'zlib' is not defined

DERP. My fault. Add the line 'import zlib' to the list of imports at the beginning of bf4dumper.py. Edited November 30, 2014 by wogoodes

[wogoodes]

I've got this kinda sorta running, but I've added exception handling to skip past any error that I may run into, and I've also disabled anything having to do with CAS patches. I'm spending my time looking through what I've dumped out right now (such as Lua scripts) so I'll probably return to this later.

[nightmaremutant]

i get some problems when i try to dump bf4, tried to comment out different cats, paths seems to be right.

"Traceback (most recent call last):

File "C:\fbyte\bf4dumper.py", line 257, in <module>

if "tocRoot" in locals(): dumpRoot(tocRoot)

File "C:\fbyte\bf4dumper.py", line 248, in dumpRoot

dump(fname,targetDirectory)

File "C:\fbyte\bf4dumper.py", line 148, in dump

baseToc=cas.readToc(baseTocPath)

File "C:\fbyte\cas.py", line 95, in readToc

return Entry(unXor(tocPath))

File "C:\fbyte\cas.py", line 77, in unXor

f=open(path,"rb")

IOError: [Errno 2] No such file or directory: 'C:\\Program Files (x86)\\Origin Games\\Battlefield 4\\Data\\Win32\\XP0Vehicles.toc'"

EDIT: it seems like it tries to find a toc file where it isnt located in the normal data folder, when its located in the update folder, i have no idea how to redirect it to the correct folder

Edited February 6, 2015 by nightmaremutant