Entertainment, Sports and News Company Turner Studios Relies on Bacula Enterprise Edition

Turner Broadcasting System, Inc., a Time Warner company, is an entertainment, sports and news company that creates content and delivers experiences to fans whenever and wherever they consume content. These efforts are fueled by data driven insights and industry technology.

It owns and operates valuable brands in the world including Adult Swim, Bleacher Report, Boomerang, Cartoon Network, CNN, ELEAGUE, FilmStruck,, Great Big Story, HLN, iStreamPlanet, Super Deluxe, TBS, TCM, TNT, truTV, and Turner Sports.

Challenge
“We originally found out about Bacula through our engineering director, who at the time was looking for a more cost-effective approach to our data center’s backup, especially in view of its large amount of rapidly-growing data. The challenge was to find a low cost, hopefully open source solution, but at the same time having high quality support. We also knew that whatever backup solution we considered, it needed to be something that we could be fully in control of, meaning something that we could tweak, optimize and customize by ourselves to our specific needs,” said Andrue Netcher, technical supervisor, sportscentral engineering, Turner Studios

“As a rule, the other backup systems we looked at were too complex. They had too much of what we didn’t need, and not enough of what we did need. This problem became worse when we looked at their pricing models – it was too difficult to narrow down and take the specific pieces we needed. If we wanted just a portion of it, we couldn’t get it – we had to pay for everything.  The difficulty with many vendors was further compounded by their support models and associated costs. We simply have to have full 24/7, 365, top level support for our operation, but most vendors will make you pay a ridiculous price for that,” he said.

“One important criteria was to have a backup and recovery solution that gave us the ability to reduce our ongoing operating expenses by reducing support costs. For example, it would not have been acceptable to end up paying a vendor to come to us and modify its own code, or modify its functionality just to make it fit our system. Instead, we needed a modular system that allowed us to go ahead and do what we needed to do by ourselves,” he added.

Solution
“To be able to work closely with the Bacula developers, and also having the ability to go online to the open source forums, take their best practices and apply them to our needs was fantastic for us” said Brady Small, senior production engineer, Turner Studios. “We use Bacula Enterprise Edition to backup a 1.3PB Hitachi SAN. Thanks to Bacula’s ability to tweak it so well, we have gotten the time down to backing up that entire system in around a month. That SAN contains all our video files and those are backed up using Bacula Enterprise Edition. 600GB is our average file size, and Bacula can handle that, no problem.“

“Our team has to ensure our file based ingest, edit, and playout system operates efficiently for live, on-air production,” said Netcher. “We have to operate a mixed environment from a technical point of view. Just for our Sport digital record system, we use Windows, Linux, and Mac, we run Adobe, we run Vantage transcode servers, and we’ve got several different NAS running and multiple other SAN’s. We are now approaching the ability to simultaneously ingest 100 channels of HD content into our EVS and Harmonic ingest servers. We use Bacula Enterprise Edition specifically for backing up the SAN.“

“Regarding the implementation of Bacula, deployment went quickly. Once we decided on Bacula as a solution and got the budget approved, it only took about three months, getting the hardware in place, the networking in place and getting the software set up and running. We worked closely with Bacula’s engineering department and they gave us all the help we needed, exactly when we needed it – even when a problem was a hardware issue and not something that Bacula was responsible for. For example, Bacula Systems helped us on problems with our system infrastructure, which in turn helped us sort other, broader issues much more quickly. We are thankful to the Bacula engineers because they helped us by giving us a lot of advice and general information that allowed us to speed up the trouble shooting process – even when it concerned other problems that not directly related to Bacula“, he said.

Results
“We have a very busy IT infrastructure where there are no real off-peak hours. We have slower times between three and five am, but obviously we cannot get a full backup done in that period. When it comes to backup levels, we do fulls, incremental, and differentials. Brady throttles it sometimes in order not to hit the SAN too hard. But the most important thing is knowing that our backup solution is solid and reliable. Why? Because in our department we must take care of and safeguard video content that our company depends on for revenue. Due to the way video writes to storage, we are literally frame-to-frame dependent. We must be sure that the records are absolutely frame accurate and it is imperative that the backup solution restores to the SAN in the correct order and at the correct speed“, said Netcher.

“Small has been instrumental in finding ways to make our system operate with even more efficiency. And due to the open source nature, we have been able to share that knowledge with other Bacula customers’ IT departments to help them out, too. With Bacula Enterprise Edition, we get useful choices on which type of tool. For example, we use BWeb Management Suite – the Web GUI – for most our administration and system monitoring. But for some of the more specialized tweaking and customization we do, Brady uses Bacula’s command line interface. It is useful to be able to switch between the two. Bacula is designed to provide us with granular control over pretty much everything,” he said.

“The ability to get great performance, control the set-up and then use BWeb to go in and find the exact data we are looking for – or find a previous iteration of that specific file – has actually saved our live production on at least one occasion. We had to restore some vital video files which were accidentally deleted, including critical show elements, with very little time to spare. We were able to save the show, and that alone probably saved a month’s worth of work for different organizations throughout the company,” said Small.

Are Turner Studios satisfied with Bacula Systems’ support? “Always“, said Brady. “We are very happy with Bacula support, and we always get the help we need. Bacula’s support team always helps us find a solution.“

“As a direct result of using Bacula Enterprise Edition, we have saved money every year on support costs compared to what we would have paid with the more old-fashioned, traditional vendors. We do not get pushed from our finance department to try and reduce Bacula’s price, because they know that its charges are already very reasonable” said Netcher. “The overall advantages we get from using Bacula over other solutions are the ability to tweak it to exactly fit our needs, keeping a cap on costs and getting great support from Bacula Systems’ engineers. The ability to work with and contribute to the open source community is an added bonus for us. Next, we are looking forwards to trying Bacula Enterprise Edition’s new lloud interface feature.”