Digital Production Buzz
March 17, 2016
[Transcripts provided by Take 1 Transcription]
(Click here to listen to this show.)
BuZZ Flashback: Jonathan L
Tim Jones, President/CTO, TOLIS Group
Roger Mabon, Co-founder/CEO, MLogic
Marc Batschkus, Business Development Manager, Archiware
Larry Jordan: Tonight on The Buzz, we’re looking at archiving, both hardware and software, starting with Tim Jones. He’s the President and CTO of TOLIS Group. He’s been involved with backup and archival planning and implementation since 1982 and with tape hardware design since 1987. Tonight, he explains what we must know to safely archive our projects.
Larry Jordan: Next, Roger Mabon is the co-founder of mLogic, which designs and markets peripherals for desktop and portable computers, including Thunderbolt based computer backup and docking hardware. Tonight, Roger explains LTO tape technology and what hardware you need to successfully archive your project.
Larry Jordan: Next, Dr Marc Batschkus is the Business Development Manager for Archiware, which is a Munich based developer of data management software for backup, synchronization and archiving. Tonight, Marc looks at the software side of archiving and some options and ideas that you can use to protect your assets.
Larry Jordan: All this plus Tech Talk and a Buzz Flashback. The Buzz starts now.
Announcer #1: Digital Production Buzz is brought to you by Other World Computing at macsales.com; and by imagineproducts.com, the workflow experts.
Announcer #2: Since the dawn of digital filmmaking – authoritative – one show serves a worldwide network of media professionals – current – uniting industry experts: production, filmmakers, post production and content creators around the planet – distribution. From the media capital of the world in Los Angeles, California, the Digital Production Buzz goes live now.
Larry Jordan: And welcome to the Digital Production Buzz, the world’s longest running podcast for the creative content industry covering media production, post production and marketing around the world. Mike Horton and Randi Altman have the night off.
Larry Jordan: Tonight, we’re focusing on archiving – hardware, software and planning. Archiving is critical for the long term protection of our media assets, media that all too often can’t be reshot or recreated. But what planning is necessary? What hardware should we use? Or even what software? Our goal tonight is to help provide answers to all these questions.
Larry Jordan: But before we start, I want to remind you to subscribe to our free weekly show newsletter at digitalproductionbuzz.com. Every issue every week gives you an inside look at The Buzz, quick links to all the different segments on the show and curated articles of special interest to filmmakers. Best of all, every issue is free. I’ll be back with Tim Jones right after this.
Larry Jordan: When you’re working with media, one thing is essential – your computer needs peak performance. However, when it comes to upgrading your Mac, there are so many different options to choose from that the process can be confusing. That’s why Other World Computing carries the best upgrades that lets your computer performance and storage grow as your needs grow.
Larry Jordan: Since 1988, OWC has become one of the most trusted names in quality hardware and comprehensive support to the worldwide computer industry. With an extensive online catalog of Mac, iPhone and iPad enhancement products, as well as a dedicated team of knowledgeable experts providing first rate tech support, OWC has everything you need to take your current system to the next level. Whether you need to maximize your system’s memory, add blazing speed or enhance reliability, look no further than the friendly experts at OWC. Learn more by visiting macsales.com today.
Larry Jordan: Tim Jones is the president and CTO of TOLIS Group in Scottsdale, Arizona. He’s been involved with archive planning and implementation since 1982 and with tape hardware design since 1987. His company first released Brew, which is a software archiving product, in 1985 and a Mac version in 2003. Hello, Tim, welcome.
Tim Jones: Hi, Larry. Hello, everybody.
Larry Jordan: Tim, tonight we want to talk about archiving, so to help us set the stage how would you define the difference between a backup and an archive?
Tim Jones: That is something that comes from a number of different perspectives, depending on the type of industry that you’re in. In this industry, the two topics are very similar in that most people are backing up assets and they’re looking at storing them or keeping them for some long period, they’re not just looking at creating something that they can get back in the event of an oops.
Tim Jones: But in general terms, a backup is something that you do daily so that in the event that something untoward happens, you can restore that file tomorrow; or if you have full system… backups, you’re going to be able to recover systems that fail.
Tim Jones: An archive, on the other hand is you’re creating copies of data that are stored and kept for a long period of time. Whether for legal reasons such as medical research, or government requirements or is simply that you’re looking forward to being able to put out that 25th anniversary director’s cut of that latest new blockbuster. That’s what archrivals are about.
Tim Jones: So while the two processes tend to do the same thing initially, the purpose of the result is what differs between the two.
Larry Jordan: I was just reading your bio. You first got involved in the archiving industry back in the early ‘80s. What was it that caught your attention and got you hooked on this?
Tim Jones: I was in the coastguard at the time and I got transferred up to Boston, Massachusetts, headquarters, and basically we had a different type of computer on every desk in the office. There were IBM PCs, Wangs, Wang OISes, IBM terminals, IBM mainframes and Wang mainframes, and I got to talking to guys that were friends of mine in Washington, and we needed to come up with a way to stop that insanity and standardize everything, and one of the first solutions that we had to come up with was a way to actually back up all of that data that was on these disparate systems so that we could bring it together when we finally decided on how we were going to do it ,and that’s when I started looking at the various options for backup.
Tim Jones: Of course, back then the only thing really available was reel to reel tape and if anybody remembers the old 2400 Blackwatch tapes, you couldn’t get much better than that. They held anywhere from 34 to 60 megabytes of data and they backed it up at a whopping 112 kilobytes per second and you didn’t even have programs like Tar or CPIO, let alone some of the newer stuff. You had basically bit dumps and you dumped the data onto tape as if it were a file system and then you just kept your fingers crossed that, when the time came, you could get it back.
Tim Jones: It was an interesting time to be involved in this side of the industry, that’s for sure.
Larry Jordan: I remember those days. My floppy disk held 92 kilobytes of information. I thought I’d died and gone to heaven because the rest of them held 40 to 60, so I know exactly what you mean about limited storage capacity. But as you look at it over time, how has archiving hardware changed over the last couple of decades? Is it just faster and deeper or is there more to it than that?
Tim Jones: Bigger and faster but mainly it’s more reliable. When you hear people talk about tapes and tape failures from back in the quarter inch cartridge days, there was not a lot that we could do computer wise inside of the tape drive. All of the logic was on the computing platform. The tape drive was basically just an analog tape recorder that recorded ones and zeros instead of sweeping sideways. We tried to come up with ways to put error correction code in and that eventually grew into it. There was block redundancy in some of the early drives. The things that we did to try and make tape reliable back then, they were really reaching. At the time, it was the edge of bleeding and we bled a lot.
Larry Jordan: What tape format should we consider for archiving today?
Tim Jones: Well, today there are two primaries that we recommend. The one that everybody knows is, of course, the LTO technology. The second option, which is primarily seen in the very big system environments, is Oracle/Sun/StorageTek’s T10000 family. They are both rock solid platforms, but if you look at what the average cost of an LTO6 standalone drive is today, it’s going to run you around $3,000-$4,000 and just the drive mechanism, before you put it in the chassis that it has to go into with the T10000, starts at around 25k.
Larry Jordan: Well, LTO drives right now are made by two companies – IBM and HP. Why should we care about who packages the drive into a chassis and sells it to the end user? Because the mechanism’s the same.
Tim Jones: It all comes down to service and support. Whether you’re buying an IBM chassis, an HP chassis, a Quantum or any of those guys, QualStar, TOLIS Group for that matter, we’re all using the same hardware, the same technology in the hardware. It all boils down to service and support, so that if you’re working on a Linux box and you’re using Linux tools, you want to call a company that isn’t going to shuffle you off or tell you, “Well, can you connect it to a Windows box?” and the same thing applies for the Mac platform, which is even more problematic for some folks in that when you call some of those support teams for some of those larger organizations, they really don’t have the staff on hand or the knowledge base to support you on a Mac platform.
Tim Jones: So when you’re looking for that solution, regardless of what it is, whether it’s a standalone device or a huge room filling silo, you want to make sure that the company you’re dealing with is going to support you in your environment without having to send you all over town to find someone who knows your environment and their environment to help you out.
Larry Jordan: There are multiple levels of LTO, starting with LTO1, which is no longer manufactured. The ones that are out now are LTO5, LTO6 and LTO7. How do we decide which of those three to buy?
Tim Jones: Actually, it’s a little more wide ranging than that. We can still get LTO3 and LTO4 drives as well. In fact, we sell a lot of LTO4 drives to smaller shops that are doing things like music production and general business offs type stuff. But from the perspective of the larger capacities that we need in this industry, the 5, 6 and 7 are what you’re going to be looking at. I say, and a number of people that are in my realm agree with me, that the LTO6 is going to be the sweet spot for another two to three years.
Tim Jones: LTO7 is available in standalone. We expect to see LTO7 being demonstrated at NAB, but don’t expect to walk into NAB and walk up to a vendor and say, “I want to take three of those home,” because more than likely we’re not going to see shipping until the May/June timeframe on the automation devices. Now, you can buy a standalone drive right now, but from an automation perspective we’re looking at after NAB for availability.
Larry Jordan: Automation would be like a library where there are multiple drives ganged together?
Tim Jones: It’s less multiple drives than it is the ability to automate the tape changing mechanism. You may have a library that has 24 tape slots but it only has a single tape drive, and the reason for that is because you want to be able to start a job – a perfect example with ‘Fast and Furious 7,’ this took us over 209 LTO6 tapes to complete the backup of business, editorial, VFX, pre-vis and finals and then, because we did the daily stuff, we did that separately and that was another 53 tapes – so you can see how if somebody has to sit there and go through 261 tapes manually, that becomes a problem.
Tim Jones: Automation is what we refer to as the robotics that when you fill a tape it’ll automatically eject that tape, grab another tape and put it in for you without you having to manually have what we call a tape monkey swapping the tapes.
Larry Jordan: We’ve looked at hardware and how it’s changed over the last two decades. How has archiving software changed?
Tim Jones: There are still three primary flavors in the big field out there. In the Windows world, most of the component players utilize a format that’s called the MTF. It’s now Microsoft Tape Format, but back in 1987 when I went to work for Archive Corporation, it was called Mainstream Tape Format and it was created when we were trying to get 20 megabytes of data onto a 20 megabyte tape.
Tim Jones: Since then, coming out of the Unix world you have the Tar format, which most people are very familiar with. There’s also another Unix format that’s called CPIO, which simply is Unix language for Copy In/Out; and the newer technologies, of course Brew being one of those coming in ’85 to that realm. We had folks such as Retrospect, of course they had their own format, and now lately you’ve got the LTFS format that’s come out.
Tim Jones: In reality, a lot of the software side of things have not changed that drastically, more the hardware to support that software format. The Tar format, the specification for Tar that exists today can read the same Tar that was created years ago. The problem with Tar is you’ve got so many offshoots, there are a number of issues there where Tar is not Tar. But for the most part, Tar has stayed pretty much stable throughout time.
Tim Jones: With Brew, it’s the same type of thing. We started in ’85 and the tapes that were made, the earliest tapes we still have in our lab were made in ’88. The Brew that we have today will totally read and restore those and other platforms are doing the same type of thing. Retrospect never was one of them. It had the issue where, as you upgraded, your tapes were no longer compatible because they would find new ways to do things but unfortunately those new ways were incompatible with the old ways.
Tim Jones: LTFS has witnessed that through the five generations of LTFS that have come out, where some of the newer formats can’t read the older formats depending on which platform it was. They’ve started to sort that out and now, as we start to see LTFS3 come about supporting LTO7, we’re seeing a little bit more stability in that environment than previously.
Tim Jones: But for the most part, the software changes have been more on the user side, how the user perceives what they are doing, more than moving the bits from the disk drive to the tape.
Larry Jordan: Now, you’ve mentioned Brew. How would you describe it?
Tim Jones: We started with Brew in ’85 due to some serious failings at the NASA Satellite Division at Motorola Satellite Systems Group. Fred Fish, who has passed on now, and I worked on this a long time trying to come up with a way to make up for the shortfalls of Tar. What we found was that, Motorola being a source code licensee, Tar the application was a good application but Tar the format had holes in it that you could drive a caterpillar truck through.
Tim Jones: What we tried to do back then was come up with a way to create an archival container that, regardless of what you were writing to, whether it was tape or disk or back then you would use things like Archie and Gopher to move files from one system to another, the idea was the container had to be rock solid. It had to be something that you could look at in two hours, two weeks or two years and say, “Yes, my data is still good,” so we worked on a number of different things.
Tim Jones: We tried things like writing triplicate blocks. We came up with our own error correction code mechanism but, of course, that was coming around about the time that tape drives were starting to add read after write verification, and so we stopped that. But one thing that we did find was that in the data stream from the disk to the tape there were a number of places where bits could become non-bits and because of that we figured that the best thing we could do was put a checksum on every bit of data that we read from the file system as soon as we read it.
Tim Jones: By applying that checksum at a 2K level of granularity, meaning every 2K of data that we read off of the disk, and writing that checksum into the header for that 2K of data as it was packed into the archival container, we could then determine from the tape – whether it was immediately after the backup or ten years down the road – whether the data on that tape was still 100 percent valid. That was the fault that we saw in Tar and CPIO, they did nothing to give you the ability to verify. In fact, the only operation you had to verify one of those older formats was to restore the data to another platform and then compare the original file with the restored file to see if that worked.
Tim Jones: That just really messed up with backup windows, and even back when we were talking about hundreds of megabytes of data, the backup windows weren’t that huge, so we had to do things to shut that down.
Larry Jordan: Tim, what makes Brew unique? There are a lot of different archiving applications that are out there.
Tim Jones: Simply that, that verification mechanism, the ability for you to look at a Brew tape in two years on a completely different platform and say, “The data on this tape is 100 percent valid.” That is the big differentiator. Well, that and the fact that we also know how to talk directly to the tape drives, so while Brew is moving data between your file system, your computer and the tape drive, Brew in the command channel is actually talking to the hardware, asking the hardware continually, “Are you ok? Did that work? Are you ok? Did that work?”
Tim Jones: Because of the dual channel nature of the way we talk to the tape device, we’ve come up with a way to do that that does not forfeit performance in the IO stream. So if you’ve got an LTO6 and you’re using Brew with a MacPro, you’re going to get 170, 180 megabytes per second because of the way we talk to the device. This is also how we recognize bad devices, when other applications don’t.
Larry Jordan: Last question before we wrap up, what key points do producers need to keep in mind as they’re getting ready to archive their media? Do they just simply grab a tape and start dubbing stuff over or do they need to think about something first?
Tim Jones: There are two forms. There are the dailies and then there’s the long term. For the dailies, we live by one rule – back up everything. There is no excuse for, “Oh, we lost that to the cutting room floor.” That’s a line that should never be heard ever again in this industry. The other thing is for the long range stuff, you need to be aware of what you’re backing up. Don’t just throw it all onto a tape. Whether you’re using Brew or Tar or one of the Windows platforms or whether you’re using an LTFS based solution, you don’t want to just throw it onto tape because, while that makes it quick now, in five years how do you figure out what part of all of that you need back to do what you need to do?
Tim Jones: So we strongly urge our customers to examine their editing workflow, and their ingest workflow and come up with a backup workflow that mirrors that. So that when you’ve brought your colors team in, they’re working on the color work, back that up. You’ve got your VFX guys working on all the VFX stuff, back that up. You’ve got your editors in there chopping and chopping and chopping and you’re getting a good cut, back that up.
Tim Jones: Tape is inexpensive. Relative to the cost of a production, tape hardware is inexpensive. But what would it cost you if you needed to recreate that entire scene and you happen to have had just the perfect attitude from your actors, just the perfect sky for your lighting and just the perfect level of drizzle so you’re not having to hire a rain scaffold? That’s going to cost a lot more than that tape drive cost you.
Larry Jordan: And Tim, for people who want more information, where can they go on the web?
Tim Jones: Lots of places, but for us it’s www.tolisgroup.com and we can set you up with everything from simple tape storage to complete solutions involving digital asset management, middleware and into intake and disk.
Larry Jordan: That’s tolisgroup.com. Tim Jones is the President and CTO of TOLIS Group. Tim, thanks for joining us. Bye bye.
Tim Jones: My pleasure.
Larry Jordan: Imagine Products has been specializing in workflow applications for over 25 years. They started with Executive Producer back in 1991, an all in one logging and offline editing tool. In 2006, Image Mine was used in the Salt Lake City Winter Olympics and ShotPut Pro was released. Today, there are more than 30,000 ShotPut Pro users worldwide. PreRollPost for Macintosh was released in 2012. This is an LTFS archiving application compatible with any LTO tape drive, as well as Sony’s ODA. PreRollPost for Windows was released at the end of 2015. It’s the only LTFS archiving application of its kind for less than $500.
Larry Jordan: Imagine Products has three simple goals for its software – make it powerful, make it easy to use and make it affordable. It’s easy to see why Imagine Products has been successful for 25 years. Imagine what comes next. Visit imagineproducts.com to download a demo.
Larry Jordan: Roger Mabon is the co-founder of MLogic, which designs and markets peripheral products for desktop and portable computers, including computer backup and docking solutions using the high speed Thunderbolt interface. Prior to MLogic, Roger founded G-Technology, which is known to media folks around the world. Hello, Roger, welcome.
Roger Mabon: Hi Larry, thanks for having me on.
Larry Jordan: It’s my pleasure. I want to start with some basics. How would you describe MLogic?
Roger Mabon: MLogic is focused on, as you mentioned, storage peripherals using a Thunderbolt interface. We do have some docking solutions, some Thunderbolt expansion chassis that allows you to put cards into boxes and then hook them up to Macs, which of course only a Thunderbolt works. But our main product line at this point is archiving solutions. Thunderbolt enabled LTO systems specifically designed to archive content from hard disk based solutions.
Larry Jordan: There have been a lot of companies which have made peripherals. Why did you decide to start MLogic?
Roger Mabon: As a founder of G-Tech, I’ve sold a lot of hard drives in my day and my mission now is to actually make sure that some of that content that is residing on those drives sitting on the shelves in various post houses and what not that I see all over the place actually gets put somewhere that’s going to protect that content for the long term. Having content sitting on a hard drive on a shelf is not an archive strategy.
Larry Jordan: Why not? I want to get to that – why can’t we just leave our hard drives on a shelf?
Roger Mabon: Hard drives are not designed to sit around powered off, so having that hard drive sitting on the shelf is like bread molding, basically. You’d be lucky to get that data back in, say, five or six years. People do do it a lot. Sometimes archiving is an afterthought. We’re here to hopefully change that with our really simple and easy to use LTO solutions.
Larry Jordan: There’s a difference between making a backup and making an archive. How would you differentiate between the two?
Roger Mabon: Backup is just backing up your data for use in the near future. You’re backing up your files that you may be using on a daily basis, whatever the content might be. Archive is different. That is your actual archive for the long term. Think of it as the negative. You complete your project, you remove it from the spinning disk system that you use to actually edit and make whatever you’re doing, and then you move that content from the spinning drive to the tape system and you do this because an LTO7 tape, which is about that size and can now hold six terabytes of capacity at about $100 of cost, can sit on a shelf for 30 plus years. That’s five or six times the longevity of a spinning drive sitting around.
Larry Jordan: I want to come back to that because the 30 years is something of a misnomer. But before we talk about the tape, there’s also an optical technology that Sony is talking about – I think it’s called the ODA Architecture. What’s that compared to LTO tape?
Roger Mabon: I don’t know much about ODA. I’ve certainly heard about it. It’s an optical disk format. I think it’s based on Blu-ray type technology. As far as I recall, they have a multi disk cartridge that holds, I think, 100 gigabytes per disk and they have numerous disks in a little cartridge and the idea is similar to tape, that it will be a long term archiving strategy just using a different technology. As far as I know, it’s quite expensive and limited in terms of capacity at this point.
Larry Jordan: Well, let’s switch back to tape, which is where you and your products are focused. LTO tapes have different levels. I’ve heard of LTO4, 5, 6 and 7. What’s the difference between the different levels?
Roger Mabon: LTO has been around since generation 1, it’s been many, many years, and now we’re at generation 7, which is the big news. It’s recently been produced. LTO5, we’ll start there, because that’s when something called LTFS came out, which is an IBM software that enables a tape to be mounted as if it were a hard drive. So if you’re on a Mac and you’re using LTFS software, you actually see the tape pop up on your desktop. This is great because that enables the drag and drop functionality, which is the same paradigm as using a disk – you can just drag content to and from, which is wonderful, it makes the tape much more easy to handle and use – and that was started in generation 5.
Roger Mabon: Now, of course, we’ve moved onto 6 and now today we’re at generation 7, so…
Larry Jordan: So is the difference only in numbers or is there a performance or storage difference between them?
Roger Mabon: LTO6 has a tape cartridge that looks similar to this that handles two and a half terabytes of native storage capacity. An LTO6 tape drive’s maximum data transfer rate is 160 megabytes per second, so it’s quite fast. Moving on to LTO7, which is the tape I’m holding here, this tape now has a native capacity of six terabytes and the performance is now 300 megabytes per second, so significantly faster than a single spinning drive, which is great because that means the backup takes less time.
Larry Jordan: Can someone still buy LTO6 or is everything LTO7?
Roger Mabon: No, LTO6 is widely available. Our mTape solutions, we sell tons of LTO6. We started selling LTO7 at the beginning of this year, so both are available. Interestingly enough, what we’re finding is that most people are opting for LTO7 devices. They’re slightly more expensive but they do have the higher capacity and the higher speeds that I mentioned. The nice thing is that an LTO7 drive can read and write to an LTO6 tape. The LTO6 tapes are very affordable – about $25 for two and a half terabytes – so what we’re seeing is people opting for the 7 hardware and potentially using 6 tapes because they’re cheaper than 7s, which go for about $130.
Larry Jordan: Do they both have the same longevity and resistance to damage? Is there anything different between LTO6 and LTO7 aside from speed and capacity?
Roger Mabon: No, in terms of longevity it’s identical.
Larry Jordan: One of the things that you mentioned that I did want to come back to is the fact that LTO6 or 7 tapes have a life of about 30 to 35 years. While that’s true physically, the format itself changes about every 18 months, so can we just take an LTO6 tape, put it on the shelf and come back to it 30 years later and be able to play it? Or do we need to look at it more as an active archiving system, where we’re going to have to change our tapes every couple of formats as the tape drives change?
Roger Mabon: That’s pretty much the standard. You do want to migrate every four or five years, when the next generation might come out, or the generation after the next one. If you’re on a 5 now, it’s probably time to start thinking about moving to 7. That would be a very good thing to do. But one nice thing again is this LTFS software, which is an absolute industry standard. I can make an LTFS tape in one of our MTape units and then I can take it to any system that’s LTFS compatible, be it a Mac, Windows or Linux machine, and I could still use that tape, which is really nice. The interchange in that regard is fantastic.
Larry Jordan: One of the things I’ve heard about LTFS is that when it’s on a Mac, the Mac Finder is not tape friendly so that whenever it opens a folder on the Mac, the finder needs to record a small hidden file, which means there’s a lot of tape shuffling back and forth. Are we better off working with LTFS natively on a Mac or are we better off working with software which is designed for archiving?
Roger Mabon: That’s a very good question, one that I get asked a lot. When we sell our system, we do have a utility that will mount the tape as if there were a drive, which absolutely enables drag and drop functionality. But there are some limitations like you mentioned, especially with the Finder and Spotlight in the Mac OS, so we do highly recommend that people look at third party software packages to actually do the archiving. These packages make the process seamless and there are a lot of nice features. One of the biggest ones is verification. Obviously, when you take your source content and you put it to an LTO tape, you want to be 100 percent sure that what is now residing on the tape is exactly the same as the source. These packages do these verifications, which is really nice. They also index the footage, so when you need to go back and retrieve the footage, it simplifies everything enormously.
Tim Jones: We recommend packages from Imagine Products, one that’s called PreRollPost. There’s a company called YoYotta which makes YoYottaID LTFS software. We also work with software like Brew from TOLIS and certainly Archiware.
Larry Jordan: We’ve talked with the folks at Imagine Products on a recent show, we just finished chatting with Tim Smith over at the TOLIS group and Archiware will be on in the next segment, so we’ll be learning a lot about this. LTO drives are made by two companies, IBM and HP. Given the fact that there are only two manufacturers and lots of people packaging the drives, like MLogic, does the brand that makes that total package make a difference? Or are we really just shopping based on price?
Roger Mabon: We use IBM mechanisms internal to our products. What we’ve done is we’ve taken these industry standard drives and packaged them in a way that then allows you to attach them via the Thunderbolt connection on any Thunderbolt enabled computer. That’s our secret sauce, if you will, taking these native SAS tape drives and converting the connection to Thunderbolt. It just makes everything extremely simple, especially on Mac as all Macs have Thunderbolt connections. That’s what we’ve done and we’re actually the only Thunderbolt certified LTO, meaning Apple and Intel have certified this device. That’s basically what we do.
Larry Jordan: Let’s talk about the device itself. What is it and what does it cost? And when we’re buying it, what are we buying?
Roger Mabon: There are different flavors now because we’re supporting both LTO6 and LTO7 technology. Our desktop mTape unit with LTO6 has a retail price of $3599 and what you get with that is the desktop enclosure, obviously including the LTO6 tape drive inside, dual Thunderbolt ports, comes with a Thunderbolt cable and it comes with one piece of LTO6 media and a cleaning cartridge. That’s $3599 list.
Roger Mabon: The mTape LTO7 version is $5199 list price, so slightly more expensive, but again twice the capacity, basically twice the speed and, again, it comes with everything you need to just basically plug it in and get going.
Larry Jordan: And both these units are called what?
Roger Mabon: They’re called the mLogic mTape.
Larry Jordan: Now, you’ve also got a rack mounted version. Is that also Thunderbolt specific?
Roger Mabon: Yes, that’s correct. We have another product we call the mRack and the mRack is a 1U device rack mountable that contains one or two LTO drives. Some people opt to get two drives in this rack because it enables them, with software such as some software mentioned previously, to simultaneously write to two LTO tape cartridges for the ultimate in security, so you have a safety copy and another copy that you can put somewhere else. It’s a nice feature.
Larry Jordan: What strategy should people have when they’re archiving? What process should they go through?
Roger Mabon: Today, we have these cameras that have SSDs on them. That’s our film at this point. What people do is they take these cartridges, they offload them via some type of dock and they typically offload them to spinning drives, so that’s where you get your edit done, you do everything in color, whatever you might need to do. You finish your project and then you offload to tape for the actual archive, so once the project is finished you archive the tape. Now, that said, people are also archiving on set because these cartridges that come off the cameras are, in fact, their negatives. They’re expensive, you want to reuse them.
Roger Mabon: The nice thing about our mTape is it’s a portable little device that you can bring on set, and you can actually to the archive in the field and then you can recycle these cards for your camera, knowing that you have that on here so that you’re ok with erasing that actual footage right off those camera cards because you know it’s on LTO, where it’s the safest possible place to put your media.
Larry Jordan: Should we have more than one copy of our archive?
Roger Mabon: Yes. The reality is archiving seems to be an afterthought. We realize this, we’re trying to make this as easy and affordable as possible so it’s maybe not so much of an afterthought, but the fact is it is an afterthought and people are putting their stuff on hard drives and once you lose something you start to think about the archive strategy. Yes, it’s good to have stuff that you’re going to use in the short term sitting on hard drives. You certainly have to put it here. This is your negative, basically, and we talk about the cloud, the cloud is also an interesting place to put content, but if you’re generating these massive amounts of 4K data that we see nowadays, the cloud can get very expensive whereas for $100 I can put my six terabytes right here and I have my hands on it and I know it’s going to be around for a long time.
Larry Jordan: Roger, where can people go on the web to learn more about mTape and mRack?
Roger Mabon: It’s www.mLogic.com.
Larry Jordan: And Roger Mabon is one of the co-founders of mLogic. Roger, thanks for joining us today.
Roger Mabon: Thanks, Larry. Thanks very much.
Larry Jordan: Welcome to Tech Talk.
Larry Jordan: Let’s do something really cool. Let’s say that we had some audio recorded on set and it was noisy. We needed to re-record it, so we’ve got an ADR session. We’ve got our on set audio and we need to reconcile the ADR audio with the on set audio; and for those of you who have ever done ADR work, you know that you want to have somebody else do it because it’s just painful. Audition makes it easy. Watch this.
Larry Jordan: Let’s go up to here and say ‘close all’ – I’m not going to save any changes because I want to come back to where I left off – and let’s go to ‘file.’ Let’s open up a recent project, which is ‘Speech Align.’
Larry Jordan: Here is the original on set audio, in a track called ‘On Set.’
Recorded audio: One of our key audio challenges is getting the same talent.
Larry Jordan: Ok, then I went and I re-recorded it because I needed to have a richer sound.
Recorded audio: One of our key audio challenges is getting the same talent.
Larry Jordan: Now, if you’ve ever worked with on set audio, it’s never as good as what we’ve just listened to. There’s noise and clanking and goodness knows what. The gremlins conspire to make on set audio as bad as it possibly can be. Now I’ve got to reconcile these two, but if I play these two clips together, here’s the problem.
Recorded audio: One of our key audio challenges is getting the same talent recorded on different…
Larry Jordan: They don’t sync, so now I’ve got to adjust on a syllable by syllable basis to take this ADR and get it to lip sync with the on set audio which, as you can imagine, is not for the faint of heart nor somebody who’s in a hurry. This is a painstaking process. But watch what Audition does. I’m going to select these two clips here, then I go up to the clip menu and go to ‘Automatic Speech Alignment.’ I want to align my reference. My on set clip is called ‘Mic 2,’ we can see the name of the clip there. It’s in the track where I’m putting all of my on set audio and I want to move around the clip called ‘Mic 1’ – you can see the clip header there.
Larry Jordan: I want to have it be really tightly aligned. The default is balanced alignment and stretching, but I want to have really tight alignment because it’s going to stretch the clip, speed it up or slow it down, without changing the pitch if I ask it to. I’m going to have it add a new track, click ‘ok.’ That quickly, it’s aligned it. Now, look at the header. I’ve got one that’s ‘Mic 2,’ one that’s ‘Mic 1.’ My aligned track has been ‘Mic 1’ voiceover aligned, it’s in the middle. Let’s mute ‘Mic 1’ and let’s listen to the source clip, which is here, with the aligned clip and see how tightly aligned they are.
Recorded audio: One of our key audio challenges is getting the same talent…
Larry Jordan: Which is pretty darn amazing.
Recorded audio: …recorded on different days using different mics to sound the same.
Larry Jordan: Now, these clips didn’t even start at the same time. Yes, they were in the ballpark, but we heard before they weren’t perfectly aligned. Now, let’s take a listen to the aligned clip and see what it sounds like.
Recorded audio: One of our key audio challenges is getting the same talent recorded on different days using different mics to sound the same.
Larry Jordan: Now, the reason I chose this option is I wanted to get the alignment really tight. Let me just undo this. Select both clips. What I would do in real life is a slightly different setting. Clip, go to ‘Automatic Speech Alignment,’ and I would change this to ‘Balanced Alignment and Stretching’ because I end up with a higher quality result. Click ‘ok,’ look for the clip that’s got aligned in the title and now when I solo this and play it…
Recorded audio: One of our key audio challenges is getting the same talent recorded on different days…
Larry Jordan: Which is just amazing. If you’ve ever spent days, even weeks, trying to get ADR to work with your project, this can get you so close so quickly that all you have to do is spend time with the exceptions, not spend time with the vast majority which can be automatically aligned using Audition.
Larry Jordan: Dr Marc Batschkus is the Business Development Manager for Archiware, which is a Munich based developer for data management software for backup, synchronization and archiving. Marc’s background includes archiving, media informatics and data management. Hello, Marc, welcome.
Marc Batschkus: Hello, Larry. Thank you for having me again.
Larry Jordan: Oh, it’s my pleasure. You know, I was just thinking that getting started in data management and archiving is not a normal switch, coming out of school. What got you interested in this in the first place?
Marc Batschkus: I actually did a lot of data management and media informatics at university for a very long time and so the switch came naturally when, let’s say, the road in academics was getting too narrow.
Larry Jordan: How would you describe Archiware?
Marc Batschkus: We are a group of friends, really, working together very closely like in a big family. Attention to detail, that’s the important thing.
Larry Jordan: Archiware is a middle sized company, but what part of data management do you focus on?
Marc Batschkus: It’s cloning data for data availability, time critical stuff; then we have a routine backup product for workstations and for servers, two different products, and then we have an archive product for long term archiving. But they all scale from the very small installation – it might be a Mac Mini and a single tape drive – to medium and very large installations… really large enterprises.
Larry Jordan: Who would be a typical customer?
Marc Batschkus: A typical customer is where we have dozens or maybe even hundreds, so post production companies ranging from five to 50 people, something like that. That’s our main range. But having said that, we have one or two people companies and up to nationwide or global companies as well.
Larry Jordan: Would you say that you’re focusing more on the small customer market, the middle customer market or the enterprise market? Where’s the focus of the company?
Marc Batschkus: It’s the mid-sized market and the small enterprise market. In the enterprises, we often have island installations, so there might be a global Fortune 500 company that everybody knows and they’re our customers but we’re not doing the main data management solution, we do something for Department A and Department D and Department F and that’s our strength, because we are more accessible, easier to use and easier to maintain than the very big solutions that are used globally.
Larry Jordan: There’s lots of different archiving software and you’re not the least expensive in the market, there’s stuff that’s free and stuff that’s nearly free and Carbon Copy Cloner is probably the most famous of all of these, and there’s archiving software. What is it that’s unique about you? Why should somebody invest the money necessary to get Archiware on their system?
Marc Batschkus: First, it’s the flexibility. The platform choice is completely up to you, we are platform agnostic. We support Mac, Windows, Linux, Solaris and 3 BSD, so people can build their own appliances, people can have very high performance servers running with us, and people can choose to move from one platform to the other very easily. Basically, you copy one configuration directly over from your, let’s say, Windows server to the Linux server that you choose as the next platform and then you’re good to go and everything is already up and running.
Marc Batschkus: That makes it a very attractive platform for people who are in charge of mid to large installations, not just a company where there are two servers, but somebody who has ten, 15, 20, 50 servers to take care of where there’s a reasonable chance that they might change platform, they might have to move to another platform, stuff like that, so this versatility is important.
Marc Batschkus: Then performance wise, there’s a huge gap between those free tools and us. If you compare them, if you do identical jobs with them and with us, it’s sometimes by a factor of ten that we are faster.
Larry Jordan: I want to take a step back, because throughout today’s show we’ve been looking at the process of archiving from a hardware point of view and a software point of view, but there’s actually a process to archiving. What should media creators think about when they’re getting ready to archive their projects? What codecs should they use? How many archives should they keep and how do you deal with the future?
Marc Batschkus: When you’re planning to build an archive, you should think about the use case five years down the road, because there might not be anybody around who remembers anything of today’s production – the environment, who was involved, the codecs, all the machinery used, whatever it is. In five years from now, everybody might have changed but they would still have to access the material and find it. So how do you find it? You need metadata, so you need a metadata schema and that’s not an IT topic but an organizational topic. You have to work out what descriptions you need, what methodology, what categories, whether you can cut down on specific vocabulary to make that consistent. This is very important because this is the key to the archive later on, the metadata schema.
Marc Batschkus: Then the use case and thinking about the future – who would it be? What role would the person have? Would it be an editor? Would it be a producer? And what would this person know about the stuff that is already recorded and captured in the archive? From there, you work your way backwards and think about what you would have to put in today so that it makes sense ten years from now to work with and enable you to find stuff.
Marc Batschkus: Having said that, you always have to make a choice and it’s a balance that you have to strike for your individual production environment, on codecs, on machinery, on operating system level that you have to maintain and put on the side, even if you’re not working with it any more, to have access to tools that might be essential to reconstruct your production from today in five years, because the operating system in five years might not support the plug-in, the specific little tool that did some tweaking, the recalculation of something, whatever it might be, and so you would have to be very careful and document all the surroundings of your production to be able to track them down and conserve them for the future, the essential parts at least.
Marc Batschkus: I would suggest to media companies to always keep old machines as they are in some basement or somewhere available because they might need them in some years. Then you can look at the most future proof codecs that might make sense to choose right now to put your stuff in, but there are always specifics and there are always details hidden in there that will catch you later on if you’re not very detailed and document very well, so documenting everything well just in a text file or something is an essential part, I think.
Larry Jordan: Do we need to transcode our media, for instance convert .mp3s to .AIFFs or .WAVs or convert our video into a more future proof codec? Or are codecs going to be around for a while without changes? What would you advise?
Marc Batschkus: I would always use the highest quality codec that you can get, so .mp3 isn’t really the best one. You should record it as an .AIFF or something and keep it that way and then you just down sample it for your production needs. Keep the original files in a native condition and keep track of everything that’s necessary to access them, which might be plug-ins or tools that allow you to open your raw footage or your native recording format. The native formats not transcoded would be the things that I would archive and strongly suggest you archive so that you can always go back to the source format and transcode from there, even if you need it in five years or something.
Larry Jordan: So the key is to keep all of our camera native, our original formats, and make sure that those get retained; and then I would also suggest keeping the final output of your show where at least you’ve got the finished version, but not necessarily keep all the interim work versions. That’s what I’m hearing from you, is that true?
Marc Batschkus: That’s true, definitely. The output is definitely important. You want to reference to it. You gave it to your customer, the customer might come back and ask you for it in the original format. Production companies sometimes lose track of the last actual edit that was actually used, so you should be able to reference that to them; and the highest quality original native format that you can keep, and as much as you can keep. If it’s possible to keep everything, it makes sense in the long run. I’ve heard that many, many times, even if it’s a burden. Of course it’s a burden because it’s a lot of storage, but for people in most situations it makes sense to keep everything around. There might be a situation – repurposing, doing a new edit, or a typical case a customer comes back and says, “Now we have our 20th anniversary. Can you do something?” and if you don’t have anything additionally, that’s not good. You always want to do a little variation of something that you gave to them as their last whatever it was.
Larry Jordan: One of the things that you guys do is backup to tape so that we’re not just storing it on hard disk, but storing it to tape. What tape formats do you recommend?
Marc Batschkus: Since we came from tape 20 years ago or something and supported everything… and all those crazy formats that are no longer really around – they are still used by some people but only legacy installations – right now it’s LTO tape. It’s about LTO5, 6 and now 7, since the end of last year, and LTO tape is the only format that’s accessible to us, to all of us, the basic industry usage, so yes, it’s LTO5, 6, 7.
Marc Batschkus: LTO7 just came out with a big performance improvement, so they now have a performance of 300 megabytes per second native throughput, which is impressive, and six terabyte per cartridge native capacity, which makes it incredibly compact as a format. You have this 3½ by 3½ inch cartridge that holds six terabytes of your footage, which is really impressive. They are still a bit costly, but the prices will of course over time come down.
Larry Jordan: One of the things I’ve learned as I learn more about archiving is that we can’t just simply lay it off to tape and forget about it. We’ve got to actively manage our assets every five or six years, copy them from the old format to the new format. Is that a correct understanding?
Marc Batschkus: Yes, that’s true. Every technology needs migration sooner or later. With disks, we don’t notice it that much because we buy a new MacBook or we buy a new Mac Pro or something, but that’s also a migration of the storage technology. We might have those hybrid SSD disk combinations now in there, so it is a migration of technology, or we might have only SSD storage now, so it is again a migration of the technology.
Marc Batschkus: This happens more invisibly, though we do it in front of our eyes and, with long term archiving, when you reach out and have more than two generations to the most recent LTO generation, then that’s a good time to start thinking about migration. But having said that, they’re guaranteed for ten year repurchasing for ten years. You can still buy, I think, LTO3 drives, which is really old stuff. LTO2 was taken out of this repurchasing guarantee recently, I think last year, but LTO3 is still available as a drive and, of course, the media.
Marc Batschkus: The LTO consortium takes good care that stuff is available for a very long time, but still if you’re now on LTO3 it is high time to migrate to LTO5 or 6.
Larry Jordan: As we wrap up, what top one, two or three things should people keep in mind as they’re planning their own archives? What’s the key take home that you want people to remember?
Marc Batschkus: Take time for planning and involve all people that even remotely are in touch with it, all the stages of the production. Involve all the people because everybody in the end can contribute a little bit and the planning process is so crucial. It’s hard to re-implement something that you forgot two or three years ago. There might be a lot of relabeling or retagging of media and that’s a waste of time. You can really solve all of that when you have a thorough planning process, get everyone on the same page, round one table, discuss it for a number of weeks or months and put it on paper, not just install it immediately on a system or implement it.
Marc Batschkus: Shuffle it around on paper, that’s a good perspective to have with enough distance to reflect on it and to see whether it’s really what you want, whether everything’s in there, whether everybody’s satisfied and whether it will work in the future. Then start implementing. Implementation with us is a five minute process. You install it and then input ten metadata fields. Each field takes you half a minute to implement and that’s it and then you start archiving, so that’s not the big thing. The big thing is the planning, thinking and making it future proof.
Larry Jordan: Marc, where can people go on the web to learn more about Archiware and your products?
Marc Batschkus: Our website is www.archiware.com and you’ll see everything about our products, P5 Synchronize, P5 Backup, P5 Backup To Go and P5 Archive and there is the new product, the P5 Archive App, which has a Mac OS X Finder integration, so it’s a one click archive, and restore is the same so there’s no additional interface, you do it from the finder. That’s the newest product we’ve just introduced.
Larry Jordan: Congratulations. Dr Marc Batschkus is the Business Development Manager for Archiware. Marc, thanks for joining us today.
Marc Batschkus: Thank you for having me, Larry.
Larry Jordan: It’s time for a Buzz Flashback. Five years ago today…
Unknown male (archive): I fell in love with a woman from Germany and I moved to Germany in 2010. When I got here, I was missing radio and I wanted to do something I’ve never done before. I wanted to do a show that has no restrictions. I decided that I would learn how to record, which I had never done before. I had only done live radio shows my entire career.
Larry Jordan: This was a Buzz Flashback.
Larry Jordan: It’s been interesting concentrating on archiving and hearing three different voices talking about the same thing and, as I think back on it, I realize the importance of planning, that we need to think about what we want to preserve.
Larry Jordan: We want to use LTO6 or later tape hardware, not because LTO4 or 5 doesn’t work, but simply because we want to make sure that the files we have can fit, because LTO6 and 7 hold the most on a single tape. We always want to keep multiple copies of each tape in different locations. Having a single tape is not enough because what happens if that tape gets lost or damaged? You want to make sure that you’ve got multiple copies of the same tape.
Larry Jordan: You want to use software that is widely used. It has the greatest chance for long term survival, so find out what other people are using and use the same software; and you also need to plan to actively manage your assets. And remember, when in doubt, preserve everything because all it costs is tape.
Larry Jordan: I want to thank our guests – Tim Jones, the President and CTO of TOLIS Group; Roger Mabon, co-founder and CEO of mLogic; and Dr Marc Batschkus, Business Development Manager for Archiware.
Larry Jordan: There’s a lot of history in our industry and it’s all posted to our website, at digitalproductionbuzz.com, and remember to sign up for our free weekly show newsletter that comes out every Friday.
Larry Jordan: Talk with us on Twitter, @DPBuZZ, and Facebook at digitalproductionbuzz.com. Our theme music is composed by Nathan Doogie Turner with additional music provided by smartsound.com. Text transcripts are provided by Take 1 Transcription – visit take1.tv to learn how they can help you.
Larry Jordan: Our Supervising Producer is Cirina Catania; our Show Producer is Debbie Price. Our engineering team is led by Brianna Murphy and includes Ed Golya and Keegan Guy. My name is Larry Jordan. Mike Horton will be back next week, as will Randi Altman. On behalf of all of us, thanks for joining us for The Digital Production Buzz.
Announcer #1: The Digital Production Buzz was brought to you by Other World Computing, providing quality hardware solutions and extensive technical support to the worldwide computer industry since 1988; and by imagineproducts.com, specializing in workflow applications for over 25 years.