Jump to content
Official BF Editor Forums
Frankelstner

File Dumper For Sbtoc Archives

Recommended Posts

Are these 127.241 all the files I should have got?

Sounds like you`re a little short.

Afaik you need to extract patched first and then overwrite with the unpatched cat, get complete version of the game files.

Share this post


Link to post
Share on other sites

Sounds like you`re a little short.

Afaik you need to extract patched first and then overwrite with the unpatched cat, get complete version of the game files.

These 127.241 files come from the original game files without DLCs. (With one pass of the dumper script)

With the other pass, I tried to extract the patched ones (Update + DLCs), but IDLE gave me "'module' object is not callable" after about 128.000 files if I remember correctly

Edited by JackusCTB

Share this post


Link to post
Share on other sites

I have used "Guid Reader" by "dainiuxxx".

guidScreen1_zps32280de8.jpg

I had select one .itexture file, and set the chunks root folder path.

Then when I pressed "Get Dds File (DXT1) 2K" I got below dds image which is incomplete. Only some portion of image extracted.

backdropteheran_01_d1_zpsb60c903b.jpg

And when I pressed "Get Dds File (DXT1) 1K" I got below image.

backdropteheran_01_d1_2_zpse5a24040.jpg

And when I pressed "Get Dds File (DXT1) 512K" I got below image.

backdropteheran_01_d1_3_zps1e252251.jpg

Can anybody please help on this.

How to get the correct dds texture file?

Share this post


Link to post
Share on other sites

oh, the compression type is correct, but the rest not.

this texture has a resolution of 1024x512, 11 mip levels...

Edited by apfelbaum

Share this post


Link to post
Share on other sites

guys

you can't assume that every texture has the same dimensions or proportions.

here is an example. not all textures are shown up because xnview doesn't support all dds compression types.

4mv5.jpg

Share this post


Link to post
Share on other sites

Dear apfelbaum,

Have you used "Guid Reader" by "dainiuxxx" to convert chunks to dds?

And have you converted all the textures one by one?

Or is there any procedure to convert all chunks to dds at once?

Please please share your procedure dude.

Share this post


Link to post
Share on other sites

sorry dude,

but i did all one by one. i glued around 98% of the textureheaders and copied the files one by one to the right folder, was quiet painful. i figured out all headers manually cause i'm not good enough to script something...

Edited by apfelbaum

Share this post


Link to post
Share on other sites

sorry dude,

but i did all one by one. i glued around 98% of the textureheaders and copied the files one by one to the right folder, was quiet painful. i figured out all headers manually cause i'm not good enough to script something...

Hats off to you buddy, that you have did that painful procedure manually one by one.

I am more novice than you, that I am still unable to understand how to get the header and glue it to the respective chunk to get the proper dds texture.

Share this post


Link to post
Share on other sites

@ Luxox,

Yes figuring out the Height & Width byte values was the easy part. As "08 00" in(HEX) = 2048 in (DEC). But please do share whatever resources you have. Thx

I`ve done some research(as much as I can comprehend) on the dds file format and it seems they are much more than just an image file (probably just stating the obvious) and while simply inserting a DXT1 header into *.chunk file works in some cases. It seems that a proper algorithm is needed to correctly convert the majority of *.chunk files.(hopefuly Kiwidog will get IceEd finished eventually) ;]

Here is my contribution for the matter, it`s just a small and primitive app that you have point to *.itexture file and then select a ..\\bundles\chunks folder, then if you click on "Get Dds file" it looks for a file in the \\chunks folder and if found copies it to the memory and merges header, then writes and new file to respective *.itexure folder.

You have to copy all 3 Dxt1HeaderTemplate_*.txt files to you ..\\chunks folder

The program is my first ever coding project so bare that in mind ;].

Guid_Reader_Converter3.jpg

https://dl.dropboxusercontent.com/u/88155050/Random%20Shiet/Guid_Reader.rar

Really really great tool.

Those textures which has DXT1 2K, DXT1 1K, DXT1 512K are working fine.

So can you please add buttons with below headers:-

(DXT5) 1024x2048

(DTX1) 1024x2048

Share this post


Link to post
Share on other sites

you have to add over 50 buttons to get all headers. there are over 25 different headers just for normalmap compressions...

OK, lets forget about normalmap headers. Only diffuse map headers would be great for now.

Can you please give me separate header samples for diffuse maps. Please :)

I will try to paste headers with trial and error method. Now I can figure out the texture resolution size by looking into respective chunk file size.

Share this post


Link to post
Share on other sites

Just some more information that I had a realization about today when working on IceEditor.

The archives that normally start with either 0x81 or 0x82 is a logical or operator.

If we look at the structures that you can pull out of the exe that look similar to.

   public enum DbObjectType : byte
   {
       Eoo = 0x00,
       InternalMin = 0x00,
       Array = 0x01,
       Object = 0x02,
       HomoArray = 0x03,
       Null = 0x04,
       ObjectId = 0x05,
       Bool = 0x06,
       String = 0x07,
       Integer = 0x08,
       Long = 0x09,
       VarInt = 0x0A,
       Float = 0x0B,
       Double = 0x0C,
       Timestamp = 0x0D,
       RecordId = 0x0E,
       Guid = 0x0F,
       SHA1 = 0x10,
       Matrix44 = 0x11,
       Vector4 = 0x12,
       Blob = 0x13,
       Attachment = 0x14,
       Timespan = 0x15,
       StringAtom = 0x16,
       TypedBlob = 0x17,
       Environment = 0x18,
       InternalMax = 0x1F,
       Mask = 0x1F,
       TaggedField = 0x40,
       Anonymous = 0x80
   }

The Anonymous field means that there is no "data name" to it. That means something like this "should" work to parse everything correctly. I still have not debugged enough to know what it means if something is not anonymous. Probably just to continue extraction like it was before with IceEditor. This was a "bug" in IceEditor that I would ignore to get the files I wanted but now that my understanding of it has changed it may mean a small step closer to a public release.

           if (p_Reader.BaseStream.Length - p_Reader.BaseStream.Position < 2)
               throw new IndexOutOfRangeException();

           // This should fix the error reading anonymous types
           byte s_typedType = p_Reader.ReadByte();
           bool s_isNamed = Convert.ToBoolean((s_typedType & ((byte)DbObjectType.Anonymous)));

           if (s_isNamed)
               s_typedType = Convert.ToByte((s_typedType & ((byte)~DbObjectType.Anonymous)));

           DbObjectType s_type = (DbObjectType)s_typedType;

           m_element = new DbObjectElement(ref p_Reader, s_type);

Like I said, I just wrote that without actually testing (I need to finish coding the rest of the classes and reverse some more DbObject classes from BF3/MOHW)

Share this post


Link to post
Share on other sites

Hey all, I'm having some questions and some things I'm having problems with trying to use the tools you peeps have provided.

Firstly, I just wanted to say that since BF4 Beta is out for Pre-load, the files are available, and I've been trying to use the tools that you guys created,

and I can say I kind of failed, since it does its thing but, changes sizes constantly, and the program stops responding?

4DNAS.png

4DNzM.png

4DNzj.png

And on a side note, Frank should try to update his sound decoder to fit the BF4 Beta files, I attempted to use the one he updated for the Pre-alpha,

but it would not work either, could anyone speak up and update me on how to use the chunk to DDS converter stuff as well?

Thanks

- Cas

Share this post


Link to post
Share on other sites

@ Luxox,

Yes figuring out the Height & Width byte values was the easy part. As "08 00" in(HEX) = 2048 in (DEC). But please do share whatever resources you have. Thx

I`ve done some research(as much as I can comprehend) on the dds file format and it seems they are much more than just an image file (probably just stating the obvious) and while simply inserting a DXT1 header into *.chunk file works in some cases. It seems that a proper algorithm is needed to correctly convert the majority of *.chunk files.(hopefuly Kiwidog will get IceEd finished eventually) ;]

Here is my contribution for the matter, it`s just a small and primitive app that you have point to *.itexture file and then select a ..\\bundles\chunks folder, then if you click on "Get Dds file" it looks for a file in the \\chunks folder and if found copies it to the memory and merges header, then writes and new file to respective *.itexure folder.

You have to copy all 3 Dxt1HeaderTemplate_*.txt files to you ..\\chunks folder

The program is my first ever coding project so bare that in mind ;].

Guid_Reader_Converter3.jpg

https://dl.dropboxusercontent.com/u/88155050/Random%20Shiet/Guid_Reader.rar

Nice, but how do you open this program?

Share this post


Link to post
Share on other sites

Hello, two questions:

-Are all the unpatched files 127.241 (bundles + chunks)?

-What's the size of "0f3703949b87c0fa9eb131b304bf91d7.chunk", 104.618.443 bytes, 1.398.128 bytes or neither?

Thanks a lot.

Share this post


Link to post
Share on other sites

Hello, two questions:

-Are all the unpatched files 127.241 (bundles + chunks)?

-What's the size of "0f3703949b87c0fa9eb131b304bf91d7.chunk", 104.618.443 bytes, 1.398.128 bytes or neither?

Thanks a lot.

1) It depends on the language of your installation. There are localization chunks in Battlefield 3\Data\Win32\Loc, so the number of files can vary slightly if you have more or less archives there. 127k files seems okay to me. I have 213k but that includes all patched files and all DLC files.

2) 1398128 is correct. The largest file I can find is a 75 MB chunk (which is the video component of an SP cutscene).

Share this post


Link to post
Share on other sites

1) It depends on the language of your installation. There are localization chunks in Battlefield 3\Data\Win32\Loc, so the number of files can vary slightly if you have more or less archives there. 127k files seems okay to me. I have 213k but that includes all patched files and all DLC files.

2) 1398128 is correct. The largest file I can find is a 75 MB chunk (which is the video component of an SP cutscene).

Thanks for the fast reply mr. Frankelstner.

For me, the largest file I have is "1e3eeb5db5c2230b59cc0e8ef9686240.chunk", with its 260 MB!

Anyway, why I can extract the vanilla files but not the patched and all DLC ones?

When I use the patched tocRoot, at about 128.000 files extracted the script gives this error: "'module' object is not callable".

Thanks a lot, again!

Edited by JackusCTB

Share this post


Link to post
Share on other sites

Thanks for the fast reply mr. Frankelstner.

For me, the largest file I have is "1e3eeb5db5c2230b59cc0e8ef9686240.chunk", with its 260 MB!

Anyway, why I can extract the vanilla files but not the patched and all DLC ones?

When I use the patched tocRoot, at about 128.000 files extracted the script gives this error: "'module' object is not callable".

Thanks a lot, again!

Oh, you are correct about that larger file. I'm not sure where I've looked before. By the way, when you rename that 1e3eeb5db5c2230b59cc0e8ef9686240.chunk to 1e3eeb5db5c2230b59cc0e8ef9686240.swf you can watch the cutscene in e.g. VLC player.

As for the error, I need the whole error message to make any sense of it. I have extracted all bf3 files including all DLCs and patches without issues.

Share this post


Link to post
Share on other sites

Oh, you are correct about that larger file. I'm not sure where I've looked before. By the way, when you rename that 1e3eeb5db5c2230b59cc0e8ef9686240.chunk to 1e3eeb5db5c2230b59cc0e8ef9686240.swf you can watch the cutscene in e.g. VLC player.

As for the error, I need the whole error message to make any sense of it. I have extracted all bf3 files including all DLCs and patches without issues.

Thanks for the funny swf cutscene thing!

Anyway, I tried running the script for these update files and it popped another error, after 7002 files:

Traceback (most recent call last):
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 400, in <module>
   main()
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 397, in main
   dump(fname,outputfolder)
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 244, in dump
   casHandlePayload(entry,chunkPathToc+hexlify(entry.elems["id"].content)+".chunk")
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 364, in casHandlePayload
   if compressed: out.write(activeCat.grabPayloadZ(catEntry))
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 131, in grabPayloadZ
   payload=zlibb(cas,entry.size)
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 71, in zlibb
   outStream.write(f.read(compressedSize))
OverflowError: Python int too large to convert to C long

Thanks again for the quick and comprensive replies!

EDIT: After another run of the script, after 127.247 update + DLC files, the 'module' error has popped

Traceback (most recent call last):
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 400, in <module>
   main()
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 397, in main
   dump(fname,outputfolder)
 File "C:\Program Files (x86)\Origin Games\Battlefield 3\Update\New folder\dumper.py", line 295, in dump
   bundle=Bundle.Bundle(sb)
TypeError: 'module' object is not callable

Thanks again!

Edited by JackusCTB

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×