So, I’ve been quietly working on Global Storage, verison 2.
Verison 1 never really went beyond a "neat thing" I made. I evaluated it for usage on Terence Malik’s "Tree of Life" and decided it wasn’t mature enough yet, to trust production assets to. I also evaluated GlusterFS and MooseFS at the same time and came to the same conclustion regarding those offereings at that time. Instead, I went with a venerable NFS cross mounting solution, leveraging the power of my network switch and the inteligence of the my artists, rather than the filesystem, to handle the workload. Once I set down ground rules, and taught my team to follow them, all was well. That soution doesn’t really scale. But it was fine for that show.
However, I’ve recenlty found myself really missing the functionality that Global Storage was mean to bring, as I’ve been working a lot in Unity3D and other digital asset heavy environments. So I have gone back in and started a rewrite from scratch. The base tennets of the system have not changed. But I decided to break backwards compatibility in order to address a lot of the "bigger facility" issues I’ve been exposed to during my time as the Director of Software and Pipeline at Method Studios.
Some major changes:
- All SSH back end and communication for security
- Bazaar rather than Subversion as a base level revsion system. This allows a more distributed asset managment system just like Bazaar allows a more distributed development system than Subverion does.
- Virtual File System via FUSE. Looks exactly like a native file system but, on closer inspection, it isn’t. Its way more.
That last item, the VFS is one that I may hold off until version 2.5. Its designed with it in mind. Its just that I may finish everything up but that and push it into use using symlinks as a glue mechanism before I dive into the FUSE module. We’ll see.
Currenlty Global Storage 2 (a.k.a. gls) is awake. It can log into a minimally configured server and set up some base level storage buckets. It can manage SSH and RSA passwordless login for a user (somewhat important to a system premised entirely on SSH network commuincations). I’m implementing the Bazaar storage bucket types right now actually. Hopefully it will do somethign impressive enough to show off, very soon.
I guess the other piece of this puzzle is as follows:
gls is likely going to be the basis of all my licensable technology. I’m going to use it as my distribution, licening and updating machanism.
In a lot of ways, this is turning into version 2.0 of the "Tree of Life" pipeline. I figure if I can solve the scalability and multi-site concerns of Method, along with the budget, image size and quality isues of "Tree of Life," then I’ve gotten something pretty compelling.