When I was at NAB New York back in the fall of 2019 the world was a different place. We were traveling and eating out and having trade shows and gathering for in person learning at Post-Production World. I was walking the show floor when I stopped to chat with my friends at Bridge Digital and they told me about a service called LucidLink. It was cloud-based file storage that would let you store your video and audio files and “stream” them in real-time from the cloud to use in post-production. I said, “that seems too good to be true.”
Fast forward to fall 2020 and I’ve had the opportunity to take LucidLink for a spin. A click on the LucidLink website greets you with this:
I think of this technology as another evolution in our remote/collaboration world that COVID has thrust us into. We’ve had a couple of remote video editing talks here on PVC and I was able to kick the tires on the amazing BeBop Technology platform. Add to that extensive use of Postlab, the amazing Postlab Drive, this Nextcloud product that Damian Allan looked at, and remote editing work I’ve done using various platforms and I can say there are a lot of options out there. Both BeBop and Postlab Drive use LucidLink technology.
But none are quite as unique and quite seemingly magical as LucidLink.
What exactly is LucidLink?
First I think it’s important to understand just what LucidLink is and what it gives you when it’s all set up. LucidLink will mount a volume right on your desktop that looks just like local storage or a hard drive plugged into your system. There are a lot of internet buzzwords in this review as well as when you look around the LucidLink website. Anything cloud-related likes to use lots of buzzwords.
Above is the LucidLink volume the company set up for me (named simmons) mounted on my desktop just like a hard drive next to a Thunderbolt drive, the system drive and a USB stick. It’ll be there in open and save dialog boxes just like a hard drive. Your video creation apps and things like the Premiere Pro Media Browser will see the LucidLink volume just like a hard drive. You can open a Finder/Explorer window and your LucidLink volume looks just like a hard drive. Below is my LucidLink volume in DaVinci Resolve.
It looks like a hard drive but it’s not a hard drive. It’s a cloud storage volume that lives up in the cloud.
When you copy media to your LucidLink volume that media is uploaded to the LucidLink cloud but appears to be locally stored just like a hard drive. Depending on your upload speeds it might take awhile to get that media into the cloud. More on that upload to the cloud in a bit.
When you go to access that media in the LucidLink cloud on another system then it is streamed in realtime back to your system and into whatever application you’re asking to see it. It’s not as simple as your Netflix stream as there is some voodoo going on behind the scenes to make this work as it seems almost too good to be true.
LucidLink has an FAQ on their website that answers a lot of questions but I wanted to quote the one question that will be most relevant to readers of this article:
Does your solution work for video and audio editing?
Yes, LucidLink works great with video and audio editing. You can access video and audio files directly from the cloud and utilize a variety of NLE’s like Adobe Premiere Pro, Avid Media Composer, and Blackmagic Resolve to name a few. Customers use LucidLink in their VFX and post-production workflows. Also, DAW’s like PreSonus’ StudioONE, Ableton Live, and others can easily be used with LucidLink’s technology.
LucidLink and Adobe have worked together and even did their own demo which will give you more detail on how it all works.
There are other videos worth watching in the LucidLink resources page for more about using LucidLink with Premiere Pro and other tools like Kyno. Here’s one about Media Composer:
How do you set up LucidLink?
Using LucidLink is as simple as just installing the software and setting up and account for access and billing. That local software installed is the interface between your system and the cloud. It’s that cloud part of the equation that is a bit trickier. LucidLink offers both a standard storage tier and a performance tier. You’re probably guessing that video editing would require the performance tier that uses Amazon S3. You could try standard if all you need to store are still image, small product assets and maybe audio. 🤷♂️
Once LucidLink is setup, the Filespace will connect when the software launches.
One often confusing thing in the byzantine world of cloud storage comes in the form of ingress and egress fees. LucidLink tries to simplify that a bit with “block pricing.” As of this writing one month of performance LucidLink storage runs $85 and includes one terabyte of storage as well as one TB of egress. What exactly does “egress” mean? This: the action of going out of or leaving a place. So when you’re accessing and streaming those files in LucidLink that is egress.
If you go over that one TB it costs $50/TB. There’s not yet and easy fuel gauge to monitor the amount of egress you’ve used at any given time but that could be important if you’ve got several folks working on the same project. There’s also a question of how quickly you’ll reach those egress limits if you’re constantly streaming video all day for many days vs. just downloading a file. There are plans to build in a better monitoring system for customers including better overall analytics. Currently you can see the amount of egress per Filespace in the billing panel of your portal.
I would encourage you to take a long look at the LucidLink pricing page and expect this to change very soon as they modify prices. New pricing that should be more friendly overall, especially when you’re working with a team, and those should be released soon.
This question of how egress of data works is a big one so I posed a few egress questions to LucidLink and here are a few points that were mentioned:
- You are entitled to the same amount of egress that you have stored in the Filespace at a 1:1 ratio. This is the total cumulative amount of egress out of the bucket shared across all users.
- This is purely a function of the provider – if you choose to use Wasabi as the offering (for example), you do not pay for egress. However in our tests and customer feedback, the way LucidLink works with the object store the service sees better, more consistent performance on AWS than others we have tested and it seems to be better suited for video editing workloads.
- Customers are able to see the amount of egress they are accumulating during the billing period from within the billing tab in their portal. One of our roadmap initiatives is providing additional telemetry and use stats, auditing etc.
- If you do go over the bundled amount of egress, we charge for the actual per GB amount of overage.
- The intent of our model is to try and make the cloud cost more predictable – it is not a profit center for us. We generally believe that egress costs are far higher than they need to be to cover actual bandwidth costs and instead are more of a function of forcing customers to keep their data in a given provider’s cloud and to use their native services.
Standard is $16 per TB per month. You could start out on the standard plan and upgrade to the performance plan but it isn’t as easy as a click. There is data migration involved so time and cost to that. But let’s be honest, for video work you need your storage to be as fast as you can so the performance plan would be best.
Within your plan you can setup a Filespace which is a “shared global namespace.” That’s a fancy way of saying it’s kind of a partition or bucket that contains all your media. Within that Filespace you can make individual folders and the administrator can provide read-only or read-write access to those folders. That Filespace can be mounted on your system or shared with others. Currently there can be 25 users per Filespace but that is changing with a new pricing structure.
A nice little menu item keeps an eye on your file space and the quality of your connection. That connection word is more than your internet speed that something like Speedtest.net is reporting. It’s measuring that magic internet word we hear a lot about: latency. There are factors that measure latency beyond just your internet speed. Anecdotally, I will say that on my faster connections while using LucidLink I would get better connection readings.
While there are a lot of variables when it comes to anything internet if you’ve got a fast connection and you’re close to a data center and you’re working with low resolution proxy media then you can do real world video editing right from the cloud. It’s quite cool.
That experience is going to be better the lighter weight you media is. Lighter media is easy to play off your hard drive and stream from the cloud. If you adopt an offline workflow with say, 720 ProRes Proxy files for editorial then you’ll most likely get better performance than if you try to stream that 8K RED media from the cloud. I don’t think that will surprise anyone.
There’s also a cool feature called Snapshots that will snapshot your filespace and, from what I can tell, keep a copy of that snapshot. Those snapshots can then be mounted like another volume.
I’m still learning about this powerful feature but it looks like a snapshot will freeze your filespace in time when it is created and will always be able to me mounted and used. These can be manually triggered or scheduled with some command line work. There is also a default snapshot schedule as well and I can see snapshots saving someone’s rear end so it is a nice feature. I’m not sure how that will work years from now when everyone has terabytes snapshotted because it seems like at some point that is going to be a lot of data sitting up in the cloud. But what do I know, it’s a great feature. 🤷♂️
I asked LucidLink this very question and was told that snapshots “age out.” LucidLink is not meant as a long-time storage and archive solution. You’ll still need your LTOs, bare hard drives or cloud-based archives for that.
Finding a real world use case(s)
I had played with LucidLink a bit but when it comes to reviewing products here on PVC I much prefer to put them to use in the real world. I had three opportunities to use LucidLink in real production environments and not just a kicking the tires scenario.
The documentary with multiple collaborators
That situation arose when I began a documentary edit where I worked with another editor as well as the director needing to access media for the film.
The plan was for each person to have a local copy of all the media (around 10 TBs give or take) and use Adobe Premiere Pro Productions. We would then share Dropbox as a place to distribute tons of other media (stills, audio, archival, stock) for the film. I don’t like to use Dropbox for this purpose as you either have to take up tons of space on your local drive or constantly manage moving the shared media between Dropbox and your RAID. You also have to manage the Dropbox Smart Sync and watch as people who don’t know any better continue to pile gigs and gigs and gigs of media into Dropbox that you do not need in the edit and you certainly don’t need cluttering up and taking space on your hard drive.
I don’t want to constantly sync all that Dropbox media to my local drive!
LucidLink seemed like the perfect solution as we could source media throughout the edit and place it on our shared LucidLink volume. As soon as it was uploaded from the source everyone with the LucidLink volume would instantly have access to the new media. No syncing or downloading required. As much as I love and religiously use Dropbox I hate it for sharing massive amounts of media during an edit. I don’t want to constantly sync all that media to my local drive! I think I said that a few times didn’t I? Plus, if I’m working between home and the office that means some of the Dropbox media will have different file paths and have to be relinked (I keep my home and work RAIDs with the same name). With LucidLink all those file paths stay the same for everyone.
I began by setting up a shared folder on in LucidLink Filespace. The Share management part of the LucidLink control panel is where you device what to share. All directories you have created on your LucidLink volume should be shareable.
Next I added both of the other collaborators as a user and sent them the login credentials.
Everyone involved downloaded and installed the LucidLink software (some are on Mac, some on PC). With everyone logged in and we all went to work.
A lot of the media sourced for this documentary is archival footage, still images and stock. While each of us has a mirrored RAID of all 4K interviews and BROLL that was shot there were other people constantly sourcing media. It would fall on the director to sort what was needed for the edit and he would load that into LucidLink from his location.
As mentioned above, when that new media was uploaded on his end we would all instantly see these new files in our own LucidLink volume.
How simple is that?
The Christmas show
Another use for LucidLink came what I got a call to edit pieces for a Christmas show. These were simple, music video style performances. The show was shot in 4K but the production created quarter size proxy files for the editors. Rather than go through the hassle of a physical hard drive drop-off and pickup, a big download from Box, a Dropbox sync, or the video hellhole that is Google Drive, I created a shared folder for that Christmas show in my LucidLink filespace.
The production company installed and mounted the LucidLink partition and uploaded all the proxy files. It appeared in my LucidLink folder so I then imported the footage into Premiere and went to work. That was a couple of days of shooting compressed into 1024×540 ProRes Proxy files. It was a smooth, right-from-the-cloud edit. This was working on my home connection which is about 230 Mbps down.
This Christmas show was a case where I did not copy the media to my local RAID. I left it all in LucidLink and edited right from “the cloud.”
The corporate meeting
You can also make good use of LucidLink on your own.
As a freelance editor I maintain both a full, working edit suite both at home and at my office. There are times when I might be working between both places. It might be when I can’t go into the office for one reason or another. It also might be when I decide to use my more powerful office machine for rendering and exporting while continuing the creative edit at home.
By having LucidLink installed and mounted on both systems it was an easy way to move media between systems without waiting for the Dropbox sync. Since the office has an internet connection that is twice as fast (it got upgraded over the holiday!) I was able to use those faster download speeds when retrieving media.
By logging into my office machine remotely it was like I had two systems side by side. By having duplicate media in both places I could complete the creative edit while physically being at home, save my project file to my LucidLink drive then remote in and open that at the office for output and upload. In some instances media lived only on the LucidLink drive.
The downsides of LucidLink for your video editing media storage
I was telling someone about LucidLink and this workflow we’ve been using for multiple editors on this documentary project and they said “that seems too good to be true.” There are a few downsides.
Your internet better be fast
You need fast and reliable internet for LucidLink to really work well for video editing and post-production. The faster the better. When you call up a cloud-based video file for playback in your NLE that’s a lot of data you’re asking to stream from the cloud, through the magic of the internet, through the OS and into your NLE’s windows all at once. The bigger the file the more data you’re streaming. You can see why a fast internet pipe is paramount for LucidLink to work.
Your internet better not go down
If the internet goes down you’re screwed. Imagine trying to access lots of files on the internet without an internet connection. Not hard to see that it won’t quite work well with no internet.
But it’s worth noting a featured called pin.
Pinning will force pinned files into the cache so they should still be there even if you go offline. Of course you need the internet running to pin files but it’s good to know there are some options to deal with spotty internet. Keep an eye into the future for a full offline mode.
Some of your hard drive space will need to be a cache
You might have a chunk of your hard drive taken up by some invisible LucidLink cache files. I discovered this while using DaisyDisk to recover some hard drive space on my laptop. While doing that I found a 16 GB .lucid file cache.
I wasn’t shocked by this. The app needs some way to manage the upload and download of media and deal with variances in the internet … at least that is what I’m guessing. There is also a setting in the LucidLink control panel for the Local cache size so I knew it was going this somewhere.
I dug into the hidden files for LucidLink and it’s easy to see the cache that is stored. I wondered why I had near 16 GB taken up when I have my cache set to 5 GB but that was because during the testing for this review and I had connected to multiple Filespaces, each with a 5 GB cache. The cache is constantly active so it can cycle out old data as it needs to write new data to the cache.
I would think that the LucidLink software will manage this and at some point purge the cache to reclaim some of that drive space, and that it does. But the cache deserves some further discussion. The user can set their cache location and size so for operations as heavy as video editing you could up the cache size and place it on a fast, dedicated SSD and you should see increased performance.
This is all configurable in the LucidLink control panel. With lots of video files in the cloud and a big, fast cache more of the source media you’re working with will become local as you’re working so your overall NLE responsiveness will improve. When uploading files to a LucidLink Filespace you’ll often see the upload be very fast initially but then it slows. That is the file moving partially to the cache first before hitting the slower bottleneck if your internet upload speed. There is a lot happening in the background so if you get your initial setup right you can just forget about it and let LucidLink do its thing.
You’ll have to deal with a few video editing bottlenecks you might not be used to
While I’ve been editing with Adobe Premiere Pro on two of these jobs mentioned above (Resolve on the other) I think these caveats would apply to all the NLEs depending on how they operate. You’ll most likely hit a few bottlenecks that will be noticeable when accessing files on a LucidLink volume vs having them living locally on your RAID or SSD. Here is what I’ve seen that has been most noticeable:
- If you open a bin that is full of LucidLink residing media and you want to view if by thumbnails you may have to wait a bit before all those thumbnails load. That can vary depending on how much media is in the bin and what your zoom level is for size of those thumbnails.
- If you load a long(ish) clip into your Source monitor or Viewer don’t expect to be able to scrub around it quite a buttery smooth as you would if that file was local. ProRes included since it is always buttery smooth when sitting locally.
- Hover Scrub / Skimming won’t be a fast as if the media was local.
- My office originally had much less reliable (and slower) internet than what I have at home. I could feel a difference when using LucidLink on the sketchier connection. It wasn’t unusable at all … except one time when my building internet went down for a hour or so.
Honestly I don’t think any of those things above are deal breakers at all considering what you’re doing by engaging in video editing and post-production with LucidLink. You’re streaming media right out of the cloud and editing it in realtime for goodness sake!
What about Postlab Drive?
If you’ve been paying attention to our little post-production corner of the internet you might know that Postlab Drive is an amazing part of the Postlab service that uses LucidLink technology for a very similar experience. Postlab Drive is similar but puts its own unique spin on things with some great customizations. I reviewed Postlab here on the Editblog and I’ll take a dedicated look at Postlab Drive very soon.
What else to say about LucidLink?
Honestly that’s about it. Oh, there’s this:
Yes that is 1.13 Petabytes. So I guess that means it is virtually unlimited.
A service like this is something you want to be as seamless and as invisible as possible. I think LucidLink has achieved that quite well. Having a good chunk of cloud storage sitting mounted on your desktop like a piece of locally attached storage is something that would have been a pipe-dream just a few years ago. It’s incredibly useful and the kind of thing that, once you integrate it into your workflow, you’ll wonder how you ever lived without it.
Pros
- It’s cloud-based storage that you can stream media directly from and edit. What could be a bigger pro than that?
- Dropbox-style file syncing can be a thing of the past for many use cases.
- You’ll most likely find lots of little use cases for streaming files from the cloud that you never knew you needed.
- I’ve said that it seems too good to be true but it is true and works well.
Cons
- It’s 100% reliant on the internet. As wonderful as the internet is if it goes down it ain’t gonna work so then that is a con.
- Parts of trying to really deeply understand LucidLink gets kinda geeky as you’ll see browsing the knowledge base.
- There is no easy way to see current data egress levels in the control panel. This seems like it could be quite important.
Cautions
- I’ve had a few times where my file space doesn’t want to mount when I start the computer. The connection software just spins. A quit and restart of the LucidLink software mounts it just fine.
- New pricing will be coming online soon so keep an eye out for that.
- We talked a lot about egress up above but not much about ingress. I’m assuming the ingress/upload to get the files into your file space is somewhat inconsequential. But we all know most upload speeds are much slower then download speeds so moving tons of files into a LucidLink Filespace could benefit from OS tools like Rsync or Robocopy so it’s not just drag and drop.
- You may feel an uneasiness near the end of the month when you’ve had heavy LucidLink usage and wonder… how much extra is this going to cost me?
- As you talk to your post-production colleagues about LucidLink be prepared to get a lot of questions and them saying … that seems too good to be true?
- Things that seem to good to be true often are and your experience isn’t going to be a buttery smooth as editing ProRes media on a fast desktop RAID so don’t expect it to be. But the convenance is undeniable and this type of cloud service will open up some new video editing and post-production workflows you never thought possible.