Jump to content
Official BF Editor Forums
Sign in to follow this  
Frankelstner

Cas Extractor

Recommended Posts

Note: This script here is deprecated. Use the sbtoc dumper instead: http://www.bfeditor.org/forums/index.php?showtopic=15731

This script extracts all binary text format based files from the cas archives. Save the code as a .py file in Battlefield 3\Data and run it. Everything will then end up in Battlefield 3\Data\safe with my guess at folder names and file names. Additionally it adds the sha1 hash of every file at the end of its filename. The number it will print while running is the total number of processed text files so far. It should stop after 49000. There are about 110000 files in total and extracting the other file types is equally easy. However I haven't taken a closer look at them yet and it may not be as easy to get any folder names. And I can tell you, extracting 50000 files into a single folder (or maybe 5000 in ten folders, one for each cas file) is quite messy.

Now you can take a closer look at the files I am currently working on (a.k.a. the files with nine different sections excluding the header).

#needs Python 2.x

import string
import binascii
import sys
import os
import struct
from cStringIO import StringIO

#cas_01.cas to cas_10.cas


def unXOR(f):
   magic=f.read(4)
   if magic not in ("\x00\xD1\xCE\x00","\x00\xD1\xCE\x01"):
       f.seek(0) #the file is not encrypted
       return f

   f.seek(296)
   magic=[ord(f.read(1)) for i in xrange(260)] #bytes 257 258 259 are not used
   data=f.read()
   f.close()
   data2=[None]*len(data) #initalize the buffer
   for i in xrange(len(data)):
       data2[i]=chr(magic[i%257]^ord(data[i])^0x7b)
   return StringIO("".join(data2))



DICE="\xCE\xD1\xB2\x0F"
cat2=open("cas.cat","rb")
cat=unXOR(cat2)
cat2.close()
def readcat(cat):
   if cat.read(16)!="NyanNyanNyanNyan":
       print "error with header"
       return

   #get file length
   cat.seek(0,2)
   catlength=cat.tell()
   cat.seek(16)


   dicecount=0 #keep track of the total number of extracted files to print it later
   while cat.tell()<catlength:
       #do the cat
       sha1=binascii.hexlify(cat.read(20))
       fileoffset=struct.unpack("<l",cat.read(4))[0]
       filesize=struct.unpack("<l",cat.read(4))[0]
       casfilenum=str(struct.unpack("<l",cat.read(4))[0])
       if len(casfilenum)==1:
           casfilenum="0"+casfilenum

       #do the cas
       cas=open("cas_"+casfilenum+".cas","rb")
       cas.seek(fileoffset)

       if cas.read(4)==DICE: ##CONSIDER MY FAVOURITE FILE TYPE ONLY
           dicecount+=1
           if dicecount%1000==0:
               print dicecount

           #find the paths in the middle of the file and make a list out of them
           pathpos=fileoffset+struct.unpack("l",cas.read(4))[0]
           cas.seek(28,1)
           pathlen=struct.unpack("<l",cas.read(4))[0]
           cas.seek(pathpos)
           paths=cas.read(pathlen)

           lastbytes=paths[-16:][::-1]
           i=0
           try:
               while lastbytes[i]=="\x00":
                   i+=1
           except:
               i=16
           pathlist=string.split(paths[:-i],"\x00")

           #find a genuine path; the first string with two slashes or more :)/>
           path=""
           pathcount=0
           for i in pathlist:
               if string.count(i,"/")>=2:
                   path=i
                   break

           #just in case
           if path=="":
               for i in pathlist:
                   if string.count(i,"/")==1:
                       path=i
                       break
           if path=="":
               path=pathlist[0]


           cas.seek(fileoffset)
           #make the folders
           try:
               separator=path.rfind("/")
               if not os.path.isdir("safe/"+path[:separator]):
                   os.makedirs("safe/"+path[:separator])
           except:
               print "error with folder creation"
               debug=open("safe/debugfile "+sha1,"wb")
               debug.write(cas.read(filesize))
               debug.close()

           #write the files
           try:
               out=open("safe/"+path+" "+sha1,"wb")
               out.write(cas.read(filesize))
               out.close()
           except:
               print "error with file creation"
               debug=open("safe/debugfile "+sha1,"wb")
               debug.write(cas.read(filesize))
               debug.close()

       cas.close()
   print "files extracted: "+str(dicecount)


readcat(cat)


cat.close()
print "done"

Edited by Frankelstner

Share this post


Link to post
Share on other sites

Originally I planned to undo the XOR encryption on the file bom.fb2 and then unzip it and use one of the files as an indicator file path names. However when comparing the possible paths I have from the files themselves and the unzipped file I sometimes get no matches at all; basically they are totally different. So in the end I still have no clue as to which path is the right one. However when going through the path list in reverse I get better results for vehicles at least. The script below saves everything in the folder "safe2".

Doesn't matter anymore because the converter script fetches correct filenames anyway.

Edited by Frankelstner

Share this post


Link to post
Share on other sites

Hai. When i'm trying to run it, it says that "invalid syntax" on line 13 - print "error with header"

What will i do?

I'm fairly sure that's because you use a newer version of Python.

Python 2.x: print "abc"

Python 3.x: print("abc")

So either adjust all parentheses around print statements or remove these print statements altogether (no harm done) or use Python 2.7.

Share this post


Link to post
Share on other sites

i am working on a new machinima and my life would be a lot easier if i had access to the soundfiles of bf3.

if been searching for a way to do this but came up empty. this here is the only place where ppl seem to have an idea if this is possible or not.

would be great if someone could help me with this. :)

Share this post


Link to post
Share on other sites

i am working on a new machinima and my life would be a lot easier if i had access to the soundfiles of bf3.

if been searching for a way to do this but came up empty. this here is the only place where ppl seem to have an idea if this is possible or not.

would be great if someone could help me with this. :)

The archives are rather easy to extract. However the actual audio is in a custom format and requires debugging which is beyond my skills.

Share this post


Link to post
Share on other sites

Not sure if anyone gives an actual fuck.

From my upcoming mod tools, a little sneak peak. I know its pretty "rigged" I plan on spending more time researching and reversing Battlefield 3 to get back at DICE for not letting me have my Mod Tools and BattleRecorder when the community asked nicely.

VenicePkg.cs

/*
* VenicePkg.cs - Mini Tool set for extracting files from the cat/cas combo package
* By: kiwidog
* Website: http://allenthinks.com
* If you want to use this shizzle right here, please contact me first (I will probably agree), Don't steal my shit people. Thats what
* kills communities. I could finish all of Battlefield 3, show off the tools and say fuck you... You wouldn't want that would you?
*/
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;

using VenicePackage.Structures;

namespace VenicePackage
{
   public class VenicePkg
   {
       private List<CatEntry> pkgEntries;

       private string catFilePath;
       private string casFolderPath;

       private BinaryReader catReader;
       private BinaryReader casReader;

       public const uint CAS_HEADER = 0xF00FCEFA; // Cas FACEOFF 

       private CatHeader catHeader;

       private uint currentOpenedCas = 0; // must be 1 or bigger

       private enum DiceFileType : uint
       {
           DICE = 0x0FB2D1CE, // DiceFile - Little Endian
           ZLIB = 0x100, // Zlib - Big Endian
           WEAPON = 0x640E0000, // Zlib - Big Endian
           UNKNOWN_01 = 0xF0270000, // Zlib - Big Endian
           UNKNOWN_02 = 0xC0790000, // Zlib - Big Endian
           UNKNOWN_03 = 0xD04B0000, // Zlib - Big Endian
           UNKNOWN_04 = 0xE09E0000, // Zlib - Big Endian
           UNKNOWN_05 = 0x1001, // ZLib - Big Endian
           UNKNOWN_06 = 0x0c000048, // idk wtf
       }

       /// <summary>
       /// Creates the readers for the .cat and .cas files and preps them for reading
       /// </summary>
       /// <param name="CatFile">Input a valid .cat file</param>
       public VenicePkg(string CatFile)
       {
           catFilePath = CatFile;
           casFolderPath = CatFile.Replace("cas.cat", "");

           catReader = new BinaryReader(new FileStream(catFilePath, FileMode.Open, FileAccess.Read));
       }

       /// <summary>
       /// Reads the contents of a valid .cat file
       /// </summary>
       public void ReadPackage()
       {
           pkgEntries = new List<CatEntry>();
           catHeader = new CatHeader();

           catHeader.nyan = catReader.ReadChars(16);

           if (new string(catHeader.nyan) != "NyanNyanNyanNyan")
               throw new Exception("Invalid .cat header");

           // Read from the catalog file
           while (catReader.BaseStream.Position < catReader.BaseStream.Length)
           {
               CatEntry catEntry = new CatEntry();
               catEntry.hash = catReader.ReadBytes(20); // Length of SHA1 Hash
               catEntry.diceFileOffset = catReader.ReadUInt32();
               catEntry.diceFileSize = catReader.ReadUInt32();
               catEntry.fileNumber = catReader.ReadUInt32();
               pkgEntries.Add(catEntry);
           }

           for (int i = 0; i < pkgEntries.Count; i++)
           {
               // We have to switch to the cas file that the offset is in. Thanks Dice! :3
               if (currentOpenedCas != pkgEntries[i].fileNumber)
               {
                   currentOpenedCas = pkgEntries[i].fileNumber;
                   if (casReader != null)
                       casReader.Close();

                   casReader = new BinaryReader(new FileStream(casFolderPath + "cas_" + currentOpenedCas.ToString("D4").Substring(2) + ".cas", FileMode.Open, FileAccess.Read));
               }

               casReader.BaseStream.Position = pkgEntries[i].diceFileOffset - 32; // Subtract the sizeof(CasEntry)
               CasEntry mCasEntry = new CasEntry();
               mCasEntry.magic = casReader.ReadUInt32();
               mCasEntry.hash = casReader.ReadBytes(20); // Length of SHA1
               mCasEntry.size = casReader.ReadUInt32();
               mCasEntry.padding = casReader.ReadUInt32();

               if (mCasEntry.magic == CAS_HEADER)
               {
                   DiceFileType mFileType = (DiceFileType)casReader.ReadUInt32();
                   casReader.BaseStream.Position -= 4; // Back Up 4 Bytes to figure out what the hell is going on

                   switch (mFileType)
                   {
                       case DiceFileType.DICE:
                           {
                               DiceFile diceFile = new DiceFile();
                               diceFile.magic = casReader.ReadUInt32();
                               diceFile.fileNameOffset = casReader.ReadUInt32();
                               diceFile.unknown_00 = casReader.ReadBytes(24);
                               diceFile.dataContainerLength = casReader.ReadUInt32();
                               casReader.BaseStream.Position = pkgEntries[i].diceFileOffset + diceFile.fileNameOffset;
                               string fileName = "";
                               char c;
                               while ((c = casReader.ReadChar()) != '\0')
                                   fileName += c;
                               break;
                           }
                       case DiceFileType.ZLIB:
                           {
                               List<ZLibFile> blocks = new List<ZLibFile>();
                               while (casReader.BaseStream.Position - pkgEntries[i].diceFileOffset < mCasEntry.size)
                               {
                                   ZLibFile tFile = new ZLibFile();
                                   tFile.magic = casReader.ReadUInt32();
                                   tFile.size = BitConverter.ToUInt32(EndianSwap(casReader.ReadBytes(4)), 0); // Big Endian Readout.
                                   casReader.BaseStream.Position += (long)tFile.size; // Instead of reading skip for now.
                                   blocks.Add(tFile);
                               }
                               break;
                           }
                       case DiceFileType.WEAPON:
                       case DiceFileType.UNKNOWN_01:
                       case DiceFileType.UNKNOWN_02:
                       case DiceFileType.UNKNOWN_03:
                       case DiceFileType.UNKNOWN_04:
                       case DiceFileType.UNKNOWN_05:
                       case DiceFileType.UNKNOWN_06:
                           break;
                       default: 
                           {
                               casReader.BaseStream.Position += 8;
                               ushort derp = casReader.ReadUInt16();
                               casReader.BaseStream.Position -= 10;
                               if (derp == 0xDA78)
                                   break;
                               // Do Nothing
                               break;
                           }
                   }
               }

           }
       }

       /// <summary>
       /// Swaps the Endian Order from Little To Big or Vice Versa
       /// </summary>
       /// <param name="input">Input Byte Array to Be Swapped</param>
       /// <returns>Swapped Byte Array</returns>
       private byte[] EndianSwap(byte[] input)
       {
           Array.Reverse(input);
           return input;
       }

       /// <summary>
       /// Cleans up the Package
       /// </summary>
       public void Close()
       {
           if (catReader != null)
               catReader.Close();
           if (casReader != null)
               casReader.Close();
           catFilePath = "";
           casFolderPath = "";
           pkgEntries.Clear();
       }
   }
}

Cat.cs

/*
* cat.cs - Cat file structures reversed soley by myself
* By: kiwidog
* Website: http://allenthinks.com
* If you want to use this shizzle right here, please contact me first (I will probably agree), Don't steal my shit people. Thats what
* kills communities. I could finish all of Battlefield 3, show off the tools and say fuck you... You wouldn't want that would you?
*/
using System;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices;

namespace VenicePackage.Structures
{
   public struct CatHeader
   {
       public char[] nyan; // "NyanNyanNyanNyan"
   }


   public struct CatEntry
   {
       public byte[] hash;
       public uint diceFileOffset; // Offset in .cas File
       public uint diceFileSize;
       public uint fileNumber;
   }
}

Cas.cs

/*
* cas.cs - Cas file structures reversed soley by myself
* By: kiwidog
* Website: http://allenthinks.com
* If you want to use this shizzle right here, please contact me first (I will probably agree), Don't steal my shit people. Thats what
* kills communities. I could finish all of Battlefield 3, show off the tools and say fuck you... You wouldn't want that would you?
*/

using System;
using System.Collections.Generic;
using System.Text;

namespace VenicePackage.Structures
{
   public struct CasEntry
   {
       public uint magic; // 0xFACE0FF0
       public byte[] hash;
       public uint size;
       public uint padding;
   }

   public struct DiceFile
   {
       public uint magic;
       public uint fileNameOffset; // Start of DiceFile + fileNameOffset = Offset
       public byte[] unknown_00;
       public uint dataContainerLength;
   }

   public struct ZLibFile
   {
       public uint magic;
       public uint size; // BIG ENDIAN BYTE ORDER!
       public byte[] data; // byte[size];
   }
}

Share this post


Link to post
Share on other sites

I give a fuck!

It's awesome to still see someone chugging away at this! :D

Please keep up the good work and keep us updated. Hopefully something comes out of this. I wish we were still able to mod the initfs_win32 file so I could up the graphics still.

Thanks for the update

Share this post


Link to post
Share on other sites
reversing Battlefield 3 to get back at DICE for not letting me have my Mod Tools

truly a noble quest! i hope you succeed, i'm pretty impressed so far.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×