Jump to content
Official BF Editor Forums
Frankelstner

File Dumper For Sbtoc Archives

Recommended Posts

I still have trouble getting it to work. Now it gives me the following error:

layout.toc
Chunks0.toc
D:/mohw_dump/chunks/d3d6ced1ace5681e6faae73f37f48d55.chunk

Traceback (most recent call last):
 File "D:\dumped\moh_w_sounds\tools\dumper.py", line 356, in <module>
   main()
 File "D:\dumped\moh_w_sounds\tools\dumper.py", line 354, in main
   dump(fname,outputfolder)
 File "D:\dumped\moh_w_sounds\tools\dumper.py", line 243, in dump
   casHandlePayload(entry,chunkPathToc+hexlify(entry.elems["id"].content)+".chunk")
 File "D:\dumped\moh_w_sounds\tools\dumper.py", line 342, in casHandlePayload
   out=open2(outPath,"wb") #don't want to create an empty file in case an error pops up
 File "D:\dumped\moh_w_sounds\tools\dumper.py", line 146, in open2
   if not os.path.isdir(folderPath): os.makedirs(folderPath)
 File "D:\Python27\lib\os.py", line 150, in makedirs
   makedirs(head, mode)
 File "D:\Python27\lib\os.py", line 150, in makedirs
   makedirs(head, mode)
 File "D:\Python27\lib\os.py", line 150, in makedirs
   makedirs(head, mode)
 File "D:\Python27\lib\os.py", line 150, in makedirs
   makedirs(head, mode)
 File "D:\Python27\lib\os.py", line 157, in makedirs
   mkdir(name, mode)
WindowsError: [Error 123] Nazwa pliku, nazwa katalogu lub składnia etykiety woluminu jest niepoprawna: u'\\\\'

The windows error says something about either the file name being wrong, or the directory name, or the syntax of the volume label.

Share this post


Link to post
Share on other sites

The issue is that mkdirs works in very strange ways when dealing with extra long path names. Of course, most files would actually work fine even within standard length (and I suppose I should make a check for string length somewhere). However, as soon as mkdirs is given a unicode path like "\\?\D:\hexing\b\c", there are two different cases:

1) The folder "hexing" exists. The script works fine.

2) The folder "hexing" does not exist. The script gives back error 123.

I'm not even sure how I would fix that in my script, but as a workaround, you simply have to make sure that the first subfolder on your hd already exists, i.e. in your case just manually create a folder named "mohw_dump".

Share this post


Link to post
Share on other sites

Updated the dumper.py to handle patched noncas bundles (e.g. found at Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Factory). The script can now handle all archives of the game.

Share this post


Link to post
Share on other sites

Updated the dumper.py to handle patched noncas bundles (e.g. found at Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Factory). The script can now handle all archives of the game.

No offense Frank but what exactly the improvement means? I have used your previous scripts, what actually can be done better now?

Share this post


Link to post
Share on other sites

Okay, so first note the difference between cas and noncas; the toc files have a flag named "cas" and if that flag exists the actual data is found in the cascat archives. The resulting sb file depends on that flag. If cas, then the sb has the same structure as the toc file and its entries give a sha1 (to look up the actual data in the cascat) and a filename/guid. The noncas sb has a completely different structure.

There are four different cases:

unpatched cas sbtoc: This is the 1.0 version of bf3. All files in Battlefield 3\Data belong to this group. These files still exist in their unpatched form because the process of patching consists of adding files in the patch folder.

unpatched noncas sbtoc: The DLC files as they are released. The Xpack folders in Battlefield 3\Update belong to this group. Once again, patches are found elsewhere.

patched cas sbtoc: The patched files for the base game without DLC. Some files in Battlefield 3\Update\Patch belong to this group. Thankfully due to cas, the differences between patched and unpatched are minimal so it was easy to handle.

patched noncas sbtoc: The patched files for the DLCs. The rest of the files in Battlefield 3\Update\Patch belong to this group (they are prefixed with XP). For these files the sb looks similar to the unpatched noncas with the exception of a few bytes at the start. The actual bundle used by the game is obtained by using the info from these bytes to cut pieces out of the unpatched sb and glue them together with the pieces of the patched sb. While my script first creates a new final bundle in memory before actually reading the content, I'm sure DICE does something more elegant like having a special file handle which simply knows whether to read from patched or unpatched when given an offset.

Edited by Frankelstner

Share this post


Link to post
Share on other sites

Updated the dumper.py to handle patched noncas bundles (e.g. found at Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Factory). The script can now handle all archives of the game.

Hello dude

what did you mean ?

You we should change this line

##unpatched: C:\Program Files (x86)\Origin Games\Battlefield 3\Update\Xpack2\Data\Win32\Levels\XP2_Palace\XP2_Palace.sb/toc

##patched: C:\Program Files (x86)\Origin Games\Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Palace\XP2_Palace.sb/toc

to

Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Factory

thanks if my question is stupidly but I am english (:D)

Share this post


Link to post
Share on other sites
Hello dude

what did you mean ?

You we should change this line

##unpatched: C:\Program Files (x86)\Origin Games\Battlefield 3\Update\Xpack2\Data\Win32\Levels\XP2_Palace\XP2_Palace.sb/toc

##patched: C:\Program Files (x86)\Origin Games\Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Palace\XP2_Palace.sb/toc

to

Battlefield 3\Update\Patch\Data\Win32\Levels\XP2_Factory

thanks if my question is stupidly but I am english (:D/>)

You do not need to modify anything in the script below

#####################################

#####################################

They are just comments I have made. If you want to extract all files in the game, you need to run the script with tocRoot=r"C:\Program Files (x86)\Origin Games\Battlefield 3\Update", then run the script again with tocRoot=r"C:\Program Files (x86)\Origin Games\Battlefield 3\Data\Win32". The # symbol marks a comment, so everything to the right of it is a comment and not used by the script.

Share this post


Link to post
Share on other sites

You do not need to modify anything in the script below

#####################################

#####################################

They are just comments I have made. If you want to extract all files in the game, you need to run the script with tocRoot=r"C:\Program Files (x86)\Origin Games\Battlefield 3\Update", then run the script again with tocRoot=r"C:\Program Files (x86)\Origin Games\Battlefield 3\Data\Win32". The # symbol marks a comment, so everything to the right of it is a comment and not used by the script.

got it bro.I owe you one.thanks a lot :wub:

I successfully extract all files

Share this post


Link to post
Share on other sites

Ok, I finally figured out how to convert the textures with some effort.

As example I use the texture:

...\res\levels\mp_subway\terrain\textures\mp_subway_grass_01_c.itexture

As Frankelstner mentioned before the guid of the texture is specified in the .itexture format.

pq6h.jpg

The marked bytes represent the chunk file in the folder:

...\bundles

If you found the file you have to glue the 128 byte dds header. It is really easy to figure out the header (which specifies the DXT compression/texture resolution...) by looking at the chunk size.

In this case the chunk has a size of 2.731 kb which is equal to a 2048x2048 DXT1 compressed texture.

kut8.jpg

This way it is possible to convert most of the textures. Maybe someone has the motivation to write a converter

regards

Edited by apfelbaum

Share this post


Link to post
Share on other sites

Ok, I finally figured out how to convert the textures with some effort.

As example I use the texture:

...\res\levels\mp_subway\terrain\textures\mp_subway_grass_01_c.itexture

As Frankelstner mentioned before the guid of the texture is specified in the .itexture format.

[img_thumb]http://imageshack.us/a/img21/614/pq6h.jpg[/img_thumb]

The marked bytes represent the chunk file in the folder:

...\bundles

If you found the file you have to glue the 128 byte dds header. It is really easy to figure out the header (which specifies the DXT compression/texture resolution...) by looking at the chunk size.

In this case the chunk has a size of 2.731 kb which is equal to a 2048x2048 DXT1 compressed texture.

[img_thumb]http://imageshack.us/a/img163/9797/kut8.jpg[/img_thumb]

This way it is possible to convert most of the textures. Maybe someone has the motivation to write a converter

regards

OK that particular texture works fine using your method.

51fe384c4a3c3b2debcc62059161701f_first_Mip0.jpg

But many other`s don`t work as good.

74d074dd0fd7cb7b6668146f875e9411_first_Mip2.jpg

e7fdb430679a83aefe9bc5e35e7517ed_first_Mip3.jpg

7a77ee21cb8ad272391eb14e917a28bf_first_Mip3.jpg

yes the last two are BF4`s

Edited by dainiuxxx

Share this post


Link to post
Share on other sites

OK that particular texture works fine using your method.

51fe384c4a3c3b2debcc62059161701f_first_Mip0.jpg

But many other`s don`t work as good.

74d074dd0fd7cb7b6668146f875e9411_first_Mip2.jpg

e7fdb430679a83aefe9bc5e35e7517ed_first_Mip3.jpg

7a77ee21cb8ad272391eb14e917a28bf_first_Mip3.jpg

yes the last two are BF4`s

sure it works with every texture. the textures have the wrong header. you have to edit the .dds header depending on multiple informations. also lots of textures have completely different compression types, so dxt is wrong.

My link

Edited by apfelbaum

Share this post


Link to post
Share on other sites

@apfelbaum

So you determine DDS header from chunk file`s size? If you don`t mind please explain?

the diffuse and specular texture use the DTX1 dds header , the alpha use DTX5 (hair, mask textures, some clothcamo), but I don't know the header in some normal maps.

now... for the maps resolution watch this:

5pih.png

some time ago I found the way for compile Bf3 textures

o8hw.jpg

if someone want the files, I can provide a link.

Share this post


Link to post
Share on other sites

Hello Frankelstner, I discovered this amazing thread but I need help to get the files in these folder:

"C:\hexing\bf3 dump\bundles\ebx\sound"

"C:\hexing\bf3 dump\bundles\chunks"

When running the dumper.py script, It extract 566 .chunk files then this error pop ups:

Traceback (most recent call last):

File "C:\Users\Jack\Downloads\fb2audio\dumper.py", line 409, in <module>

main()

File "C:\Users\Jack\Downloads\fb2audio\dumper.py", line 406, in main

dump(fname,outputfolder)

File "C:\Users\Jack\Downloads\fb2audio\dumper.py", line 253, in dump

casHandlePayload(entry,chunkPathToc+hexlify(entry.elems["id"].content)+".chunk")

File "C:\Users\Jack\Downloads\fb2audio\dumper.py", line 373, in casHandlePayload

if compressed: out.write(activeCat.grabPayloadZ(catEntry))

File "C:\Users\Jack\Downloads\fb2audio\dumper.py", line 140, in grabPayloadZ

payload=zlibb(cas,entry.size)

File "C:\Users\Jack\Downloads\fb2audio\dumper.py", line 80, in zlibb

outStream.write(f.read(compressedSize))

OverflowError: Python int too large to convert to C long

Using Python 2.7.5:

Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32

Thanks in advance!

Edited by JackusCTB

Share this post


Link to post
Share on other sites

if someone want the files, I can provide a link.

@ Luxox,

Yes figuring out the Height & Width byte values was the easy part. As "08 00" in(HEX) = 2048 in (DEC). But please do share whatever resources you have. Thx

I`ve done some research(as much as I can comprehend) on the dds file format and it seems they are much more than just an image file (probably just stating the obvious) and while simply inserting a DXT1 header into *.chunk file works in some cases. It seems that a proper algorithm is needed to correctly convert the majority of *.chunk files.(hopefuly Kiwidog will get IceEd finished eventually) ;]

Here is my contribution for the matter, it`s just a small and primitive app that you have point to *.itexture file and then select a ..\\bundles\chunks folder, then if you click on "Get Dds file" it looks for a file in the \\chunks folder and if found copies it to the memory and merges header, then writes and new file to respective *.itexure folder.

You have to copy all 3 Dxt1HeaderTemplate_*.txt files to you ..\\chunks folder

The program is my first ever coding project so bare that in mind ;].

Guid_Reader_Converter3.jpg

https://dl.dropboxusercontent.com/u/88155050/Random%20Shiet/Guid_Reader.rar

Edited by dainiuxxx

Share this post


Link to post
Share on other sites

Exuse me, I can't believe that on a 3770K with 16GB RAM dumper.py always crashes: Overflow error, Python int too large to convert to C long or MemoryError. What can the problem be? I even converted the py in pyc, the py in exe but nothing. Anyway, amazing script..... only if it worked 10 times on 10... :)

Edited by JackusCTB

Share this post


Link to post
Share on other sites

bump

Make sure you are using the right scripts and python files and in the right way. Dozens of ppl have extracted it with no problems, so the script is solid. If and if it still does not work try it on another machine. Maybe your "Uber Rig" is taking a dump... ;)

Share this post


Link to post
Share on other sites

Make sure you are using the right scripts and python files and in the right way. Dozens of ppl have extracted it with no problems, so the script is solid. If and if it still does not work try it on another machine. Maybe your "Uber Rig" is taking a dump... ;)/>

Not to sound boastful, but my rig is really that one. I've got it since december, 2012 with no problems. Anyway, to be sure, I uploaded the script(s) I'm using here.

That's how I use the script:

1. Right click on dumper.py

2. Open with IDLE editor

3. Press F5..

The script extracted only once flawlessly the original (unpatched) game files.

Share this post


Link to post
Share on other sites

Make sure you are using the right scripts and python files and in the right way. Dozens of ppl have extracted it with no problems, so the script is solid. If and if it still does not work try it on another machine. Maybe your "Uber Rig" is taking a dump... ;)/>/>

Not to sound boastful, but my rig is really that one. I've got it since december, 2012 with no problems. Anyway, to be sure, I uploaded the script(s) I'm using here

That's how I use the script:

1. Right click on dumper.py

2. Open with IDLE editor

3. Press F5..

The script extracted only once flawlessly the original (unpatched) game files.

What do you think?

Edited by JackusCTB

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×