1 Jun
2019
1 Jun
'19
1:07 a.m.
On 5/30/19 6:49 AM, David wrote:
I think it was BSD 4.1 that added quotas to the disk
system, and I was just wondering if anyone ever used them, in academia or industry. As a
user and an admin I never used this and, while I thought it was interesting, just figured
that the users would sort it out amongst themselves. Which they mostly did.
So, anyone ever use this feature?
Lots of interesting insights/stories on this thread so figured i'd throw
my hat in the ring and share a business anecdote...
For quite a while i worked in the special effects/animation industry
where fortunately (for me) unix has a long and interesting history. One
secret abut the VFX world is that it's tremendously expensive with
relatively little financial upside. in my experience it is the studios
who get most of the residual income from a blockbuster feature. also a
crew for a AAA feature requires lots of human power, computers and
storage. my shop frequently had 3-5 features in full production mode at
the same time so we were redlining all of our systems 24/7.
So aside from the cost of maintaining a large renderfarm, unix/linux 3d
workstations, editing bays etc we also had an enormous NetApp footprint
to support all these systems. Now artists love creating lots and lots
of high resolution images, and if they had their way there were be
unlimited storage so they'd never have to archive a shot to tape in the
event they need to reference it later. But obviously that's not
reasonable from a financial perspective.
Our solution was to make heavy use of storage quotas in our environment,
and then leverage quotas to provide per-department billing. An
individual user was given something like 1GB by default (here they had
their mailbox, source-code, scripts etc), then the show they were booked
on was then given an allocation of say 1TB of storage. The show would
then carve out this allocation in a per-shot basis. This allowed us as
an organization to actually keep pretty detailed records on our costs
and unfortunately isn't something I've seen replicated well at lots of
startups flush with cash these days.
I was briefly on the team responsible for managing these quotas for a
show and it was seriously an around the clock operation to keep our
disks from filling up. One of the tricks was to figure out how much
space a rendered sequence of images would consume, factor in the
time-to-render a frame and attempt to line up your backup jobs to free
up enough space so the render nodes could write out the images to NFS.
-pete
--
Pete Wright
pete(a)nomadlogic.org
@nomadlogicLA