Jump to content


Photo

BSA Archive Flags


  • Please log in to reply
11 replies to this topic

#1 fireundubh

fireundubh

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 350 posts

Posted 01 June 2014 - 08:19 AM

I used my powers of awesomeness to determine the archive flags that were used to pack the official BSAs. This information could be important!

Skyrim - Animations.bsa
Flags: Compressed

Skyrim - Interface.bsa
Uncompressed, Flags: None

Skyrim - Meshes.bsa
Flags: Compressed, Retain Strings During Startup
Resource Types: Meshes

Skyrim - Misc.bsa
Uncompressed, Flags: Retain File Names

Skyrim - Shaders.bsa
Uncompressed, Flags: None

Skyrim - Sounds.bsa
Uncompressed, Flags: Retain File Names
Resource Types: Sounds, Voices

Skyrim - Textures.bsa
Flags: Compressed, Embed File Names
Resource Types: Textures

Skyrim - Voices.bsa
Uncompressed, Flags: None
Resource Types: Sounds, Voices

Skyrim - VoicesExtra.bsa
Uncompressed, Flags: None
Resource Types: Sounds, Voices

Dawnguard.bsa
Uncompressed, Flags: Retain File Names, Retain Strings During Startup
Resource Types: Meshes, Sounds, Textures, Voices

HearthFires.bsa
Uncompressed, Flags: Retain File Names, Retain Strings During Startup
Resource Types: Meshes, Sounds, Textures, Voices

Dragonborn.bsa
Uncompressed, Flags: Retain File Names, Retain Strings During Startup
Resource Types: Meshes, Sounds, Textures, Voices

HighResTexturePack01.bsa
Flags: Compressed, Embed File Names
Resource Types: Textures

HighResTexturePack02.bsa
Flags: Compressed, Embed File Names
Resource Types: Textures

HighResTexturePack03.bsa
Flags: Compressed, Embed File Names
Resource Types: Textures

Note:
By default, all BSAs have these flags: Include Directory Names and Include File Names. You can't change those flags, so I didn't add them to the list.
 
ARCHIVE FLAGS

 
RESOURCE FLAGS

Edited by z929669, 08 December 2014 - 04:26 PM.

  • 0

#2 DocClox

DocClox

    Guard

  • Members
  • PipPip
  • 121 posts

Posted 01 June 2014 - 09:29 AM

Interesting. Do you know the difference between "embed" and "retain" as regards filenames?


  • 0

#3 fireundubh

fireundubh

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 350 posts

Posted 01 June 2014 - 09:38 AM

edit: Nevermind.

Edited by fireundubh, 01 June 2014 - 09:30 PM.

  • 0

#4 EssArrBee

EssArrBee

    Incompatibilism Manager

  • STEP Staff
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 7,721 posts

Posted 01 June 2014 - 12:56 PM

So lossy textures are okay to be compressed a second time, but lossy/lossless sound files are not okay to be compressed a second time. That is weird one. Might be from higher compression that lossy sound files usually have over lossy textures. Nice analysis, firebro!



#5 Kelmych

Kelmych

    Dragon King

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 3,975 posts

Posted 01 June 2014 - 02:57 PM

Thanks.  It helps explain why BSAopt/DDSopt has had problems for the past 2 years or so when packing BSAs (compression needs to be set manually in BSAopt/DDSopt), why Archive.exe is sometimes so fast when archiving, and perhaps what the resource type checkboxes in Archive.exe are used for.



#6 fireundubh

fireundubh

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 350 posts

Posted 01 June 2014 - 09:30 PM

Thanks.  It helps explain why BSAopt/DDSopt has had problems for the past 2 years or so when packing BSAs (compression needs to be set manually in BSAopt/DDSopt), why Archive.exe is sometimes so fast when archiving, and perhaps what the resource type checkboxes in Archive.exe are used for.

BSAopt can produce only uncompressed (0x03) and compressed BSAs (0x07). No flags. They are otherwise identical to Archive-created uncompressed and compressed BSAs.

Edited by fireundubh, 01 June 2014 - 09:43 PM.

  • 0

#7 fireundubh

fireundubh

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 350 posts

Posted 02 June 2014 - 03:03 AM

I updated the list with which BSAs have which resource checkboxes checked. I also included a table with every combination of resource. My powers of awesomeness are growing.

Edited by fireundubh, 02 June 2014 - 03:03 AM.

  • 0

#8 DocClox

DocClox

    Guard

  • Members
  • PipPip
  • 121 posts

Posted 02 June 2014 - 03:11 AM

Hmmm... When I repacked the DDSOpted textures I used "retain" rather than "embed". They seem to work ok ... but then my game's not been entirely stable this weekend ... for what I assumed were unrelated reasons.

 

Even if I repack them, it's going to be hard to say that made a difference...


  • 0

#9 Octopuss

Octopuss

    Swears by everything

  • Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1,627 posts

Posted 02 June 2014 - 03:30 AM

Could this knowledge lead to being able to repack the DLC BSAs with optimized textures instead of having those as loose files?


  • 0

#10 fireundubh

fireundubh

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 350 posts

Posted 02 June 2014 - 07:24 AM

Hmmm... When I repacked the DDSOpted textures I used "retain" rather than "embed". They seem to work ok ... but then my game's not been entirely stable this weekend ... for what I assumed were unrelated reasons. Even if I repack them, it's going to be hard to say that made a difference...

When I compare two identical BSAs, one with Retain File Names and one with Embed File Names, in a hex editor, I don't see a material difference. There's nothing different except the flag bytes. I also don't see a material difference between a plain BSA and one with Retain File Names, Retain Directory Names, or Retain File Name Offsets. It's possible that these flags actually do nothing! But with flags like Retain File Name Offsets and Retain Strings During Startup, it's more likely that these flags instruct the game engine on how to load game data from the associated BSAs. Compress Archive and Xbox 360 Archive are probably the only flags that materially change the file. If we could get tangible results from these flags—results that we can test—we could use BSAopt to create pure BSAs and manually flag them with a hex editor to see what happens. If we could produce good results, someone could make a program that "fixes" BSAopt-created BSAs or even a program that goes through your mods and automatically flags them.  

Could this knowledge lead to being able to repack the DLC BSAs with optimized textures instead of having those as loose files?

Yes? Try and report back.

Edited by fireundubh, 02 June 2014 - 07:34 AM.

  • 0

#11 fireundubh

fireundubh

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 350 posts

Posted 02 December 2014 - 01:41 AM

Fixed the post-site-crash formatting.


  • 0

#12 x79

x79

    Prisoner

  • Members
  • 6 posts

Posted 06 September 2015 - 07:44 AM

I realize I'm a little late to this party but I feel compelled to revive this. I have been repacking  bsas for a while now, including vanilla and official DLCs, and I am really surprised to hear of others having so much trouble with it. Last time I played, I combined all vanilla, update, DG, HF, and DB into 5 (I think) bsas and played for around 100 hrs with no problems. At first I had to play around with settings and figure out what could or could not be compressed but I always use max compression and I have yet to have any problems with it. I also have not observed any difference between having those original flags set or not.

The only problem I have had with compression are sound files. Xwm files don't comress well so it doesn't make sense to try anyway. Wav files do compress well but the game either crashes or won't load them consistently. AFAICT, only sound fx must be wav.

BSAOpt can create mixed archives (meaning compress + uncomrpessed). Here is an example of what I do using Convenient Horses v5.0:
  • Extract BSA (97.5MB)
  • Convert all wavs in sound/voice to xwm
    for /r %n in ("*.wav") do xwmaencode "%n" "%~pnn.xwm"
    (CD to the extraced files first! This command will recurse all folders under the currend directory!)
  • Remove original wav files
    for /r %n in ("*.xwm") do del "%~pnn.wav"
    (This will only delete a wav if there is an xwm with the same name.)
  • Open BSAOpt, choose CH files and output bsa name
  • Set compression to maximum
  • Make sure recursive is checked, select sound, uncheck all
  • Select scripts\source, uncheck all
  • Pack
  • When done, choose root folder in top treeview, uncheck all
  • Choose sound folder, check all
  • Set compression to none
  • Pack (49.1MB)
I am currently trying to work up the motivation to write my own BSA packer. I want it to be a drop in replacement for archive.exe with more options e.g. choose compression and flag options before packing, 7-zip and zopfli compression algorithms, search for duplicates before adding to the archive, automatically create multiple archives if the archive becomes too large (IIRC, you can have 2GB + 1 file). I've been thinking about it for some time now but I only just started it and I tend to start projects that I never finish.


BSAopt can produce only uncompressed (0x03) and compressed BSAs (0x07). No flags. They are otherwise identical to Archive-created uncompressed and compressed BSAs.


It may be specific to v2 but BSAOpt does set flags for me. For "Convenient Horses.bsa" the set flags are Meshes, Textures, Sounds, Voices, Shaders, and Misc.

Edited by x79, 06 September 2015 - 08:40 AM.

  • 1


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users