bevy_asset_rework.md

Hey everyone!

I've started this chat because its time to make a decision about the future of Bevy Assets and I wanted the opinions of a few trusted contributors / experts before starting a public conversation.

As many of you are probably aware, Bevy's current asset system has a number of gaps:

• There is no concept of "importing" optimized assets. "source" assets aren't always in a format desirable for a given platform target. They might be too big, slow to parse, etc.
• Assets cannot have dependencies. Something like a "scene" asset might rely on a number of other assets. These should also be loaded when the "scene" loads.
• AssetLoaders can only return one asset type. This makes loading assets like GLTF files very difficult because they can have multiple meshes, textures, scenes, etc.
• Assets are naively re-loaded every time "asset_server.load()" is called
• Assets should have per-asset user-configurable settings. Ex: what sampler should this Texture use?
• Asset lifetimes are manually managed. Assets are basically the perfect use case for ref counting and most modern engines employ this technique. Bevy Assets should ref count!

How are we going to solve all of these problems?

I would like us to consider two choices: integrate Atelier Assets or extend Bevy Assets.

Atelier Assets

Kae has been building atelier-assets for awhile now. Atelier solves all of the problems above (and more!). For a full list of features, check out the readme here: https://github.com/amethyst/atelier-assets

In a nutshell it is set up like this (Kae please correct me if I bungle the details here):

• A "daemon" capnproto RPC server runs in the background and listens for source file changes on the filesystem
• When a new/changed asset is detected, it runs an Importer on the asset, which produces serde-serializable assets. These are persisted to an LMDB database, alongside metadata like the source hash, version, settings, and asset UUIDs.
• A client (aka a game) at some point connects to the RPC server and sends a load request. The server retrieves asset metadata from the database and sends it back to the client, which then deserializes the asset and inserts it into the game's asset storage of choice.

Atelier has a lot more functionality than what I just described, so I encourage you to check out the repo / examples if you want a better understanding.

Kae has a WIP bevy_atelier integration crate here: https://github.com/kabergstrom/bevy/tree/master/crates/bevy_atelier

Bevy Assets Rework

I wanted to be as informed as possible about the space / the problems we needed to solve, so as I was exploring Atelier Assets and reviewing the code, I decided to build similar features into Bevy Assets. This took more time, but I now think I understand the space much better. "Pro-grade" asset management is way more complicated than I expected! Kae was an instrumental part of this learning process. He really groks this space and was very patient with me as I got caught up.

https://github.com/cart/bevy/tree/asset_rework

This branch has generally workable (but in some cases rough) implementations of most of the features above. It also has a GLTF loader that loads GLTFs as Bevy Scenes (with any number of meshes and textures inside). I think the design / architecture is different enough from Atelier Assets that its worth seriously considering as an alternative option. But I do want to make it clear that this is largely a different permutation of the good ideas in Atelier and almost every idea came directly or indirectly from Kae :)

Here is a breakdown of the architecture:

• AssetIo
  • a "virtual filesystem" interface. basically CRUD operations on a relative_path -> Vec<u8> map.
• AssetPath
  • A "relative path" + an optional "label" that uniquely identifies an asset
  • ex: An asset path might be first_level/scene.gltf#Mesh0 where first_level/scene.gltf is the "path" and Mesh0 is the label.
  • All loaded asset files must have a "default" asset, which is available at the root without a label. In the example above this would be first_level/scene.gltf and it would reference a top-level Scene asset.
• Handle<T> / HandleUntyped
  • typed/untyped refcounted handles. When dropped, they will free the relevant asset from its corresponding Assets<T> collection
  • Internally identified by enum HandleId { AssetPath(AssetPath), Id(TypeUuid, u64)}. HandleId::AssetPath is for assets loaded by the AssetServer. HandleId::Id is for assets created at runtime outside of the asset server.
• Assets<T>
  • Simple typed collections of assets (basically a Handle->T hashmap)
• AssetLoader
  • Takes the bytes returned from AssetIo::load_path(path) -> Vec<u8> and returns one or more in-memory assets of varying type (ex: Mesh, Texture, Scene, etc).
• AssetSerializer
  • Takes an in-memory asset (ex: a Mesh) and serializes it to a Vec<u8>, which can then be saved by AssetIo.
  • These can be registered as "importers" on AssetLoaders. When registered, any assets returned by an AssetLoader that match the AssetSerializer's asset type will be serialized and saved to the .import folder.
  • Doesn't use serde directly, but its easy to use serde inside an implementation
• AssetServer:
  • manages in-memory asset metadata, processes import/load requests using bevy_tasks, tracks refcounts, reads/saves assets using AssetIo, reads/saves .meta files using AssetIo
  • Stores asset hashes in .meta to ensure assets aren't unnecessarily re-imported
  • Loads an asset's dependencies when it is loaded
  • Handles "asset import redirects". If a user requests "image.png", but it was "imported" as a compressed "image.tex", the server will load "image.tex" as if it was "image.png"
    • Ex: We might have a PngAssetLoader that returns a single Texture in-memory asset. We could stop here and just use png as our "on-disk" format and Texture as our in-memory format, but lets say we want to optimize it for published games. We could register a CompressedImageSerializer on PngAssetLoader, which serializes the loaded Texture into a space-saving tex file. Now when users request "image.png", we redirect that request to the tex file and load it using CompressedImageAssetLoader.
  • Has a configurable "source" AssetIo (used to load assets) and an optional "import" AssetIo. When the "import" AssetIo is set, assets will be imported on load and stored using the "import" AssetIo. Here are the "common" permutations I expect to see / provide defaults for:
    • Source("assets"), Import(None): Bevy Assets behaves similarly to how it does today. Assets are not "imported", they are just directly loaded into memory from their source files / source meta. This is simpler, but it means assets aren't saved in "optimized" formats.
    • Source("assets"), Import(".import"): Would be used by editors (or run in some other background process) to produce imported assets
    • Source(".import"), Import(None): Would be used by games to consume the latest "imported" assets produced by an Editor / other background process.

Comparing Bevy to Atelier

These are the key differences I see / callouts I want to make when comparing Bevy to Atelier:

• Bevy uses Asset Paths as primary keys whereas Atelier uses UUIDs
  • Pros:
    • More natural (imo). Paths are the interface most users will interact with when loading / composing assets. Paths are also already the primary key on the filesystem. In code, they will almost universally prefer server.load("path") not server.load("0e378df0-eec9-445c-ae55-0272c41ae3b4"). In visual editors, they will select assets via a file-picker of some kind.
  • Cons:
    • Moving an asset will "break" existing paths, whereas UUIDs would remain unbroken. This will require automation to identify and fix broken paths. With Atelier, moving a dependent asset to another folder wont break existing paths. I see this behavior as "good" in some contexts and "bad" in other contexts. For example, when a user moves an asset that other assets depend on to another folder, there will be cases where they want it to continue working seamlessly (UUIDs), but there are other cases where they would want to be notified / to have hard breaks / be prompted to select a new replacement asset (Paths).
• Importing is "optional" in Bevy, whereas it is required by Atelier
  • Bevy uses AssetLoaders to map "asset sources" to runtime assets. These can be optionally "remapped" to other more efficient AssetLoaders / formats, but it isn't required.
  • Asset importers in Atelier dont return runtime assets directly. They return serialized "imported" versions of assets, which are then persisted and sent over the RPC connection and deserialized by clients.
• Bevy is "filesystem first" and Atelier is "network first"
  • Bevy uses AssetIo, a "virtual filesystem interface" that maps virtual paths to Vec<u8>. This currently has a filesystem implementation, but it could have a networked or local database backend.
  • Atelier defines an RPC interface that can be backed by anything that speaks RPC over a socket.
• Bevy is smaller and architecturally simpler than Atelier
  • Bevy is ~1.7k lines of code. Atelier is ~9.4k lines of hand-written code and ~10k lines of auto generated schema code.
  • Bevy is just an AssetServer struct with an AssetIo backend. Atelier has a web server that speaks Cap'n Proto RPC, stores metadata in a local LMDB database, and communicates with a client loader via RPC. Atelier also currently adds ~30 dependencies (largely Tokio, which Kae wants to remove, so dep count isn't a major concern).
  • There is enough code / layers containing different stacks that Atelier is significantly less "hackable" than Bevy Assets imo. The complexity of Atelier was daunting for me, and I consider myself to be pretty good at navigating codebases. This isn't a knock on Atelier. The architecture was chosen for good reasons and the code is clean. The architecture just also dictates a certain amount of complexity. I think the "hackabilty" / "approachability" of bevy is one of its more important features, so I'm a little wary of adding a system this "big".
• Bevy isn't as mature as Atelier
  • Atelier has been around for way longer, has been used in more projects, has more features, and is maintained by someone who knows way more than me about asset management.
  • Bevy Assets would need a week or so of work to get it into a merge-able / stable-ish state. Atelier is already partially integrated with Bevy thanks to Kae so we could probably get it merged much faster.
  • Bevy is also missing a couple of "high-value features" that would require a bit more time / follow-up:
    • Save/read AssetLoader settings inside the .meta files (ex: sampler configuration for textures)
    • Hot-reload .meta files
• Atelier might get adopted by Amethyst
  • It may or may not pan out, but if it does it would mean we'd have a bridge between the projects, which would be nice.

At this point I think I have clearly demonstrated bias toward the Bevy Assets rework, largely because I think its much simpler and still accomplishes most of the goals I have for an asset system. But I want to hear what you all think to protect the project from NIH syndrome and me tending to like code I wrote / fully understand.

Kae has the most context here (as the developer of Atelier Assets and my personal teacher / sounding board for Bevy Assets), so I'd like him to give you all his take first / provide corrections + additional context where needed. I will likely give his thoughts more weight than everyone elses because he really is the pro here.