Recently, doing the same project that I previously blogged about I was also asked to grab a movie off of YouTube (one banned in Cuba and only available through that medium). The quality of YouTube videos obviously is largely based on the slightly archaic codecs used by the YouTube interface to encode videos uploaded to their site. However, there are numerous “cookbook” type recommendations from many different sites for getting the best quality video to appear on YouTube. However, the creators of this particular video seem not to have read those, as the video was full of codec artifacts, dropped audio, and both of these together sometimes made for a difficult viewing experience, especially if there happened to be a lot of movement on-screen. So I searched far and wide for some possible fixes and filters to make this video just a bit higher quality.

I found that this wish of mine is largely impossible. To try and upconvert poor video to higher quality video is near impossible – otherwise filmmakers might be shooting on cell-phone cameras and saving themselves a lot of money. So here are a couple of very minimal filters (all are in FCP) one can try applying to a poor quality YouTube, or any other poor quality video to try and end up with a better result.

1. Blur Filter: This filter can obviously be taken too far, resulting in even poorer video quality. What I did find though, was that the filter cut down the pixelated look of codec artifacts when there was a lot of movement. The best way to use this filter to find a frame with particularly bad artifacts, add the filter, and play with the slider to get some of the sharp, square edges of the artifacts to go away. Move to a frame with less movement to be sure you are not blurring the general picture too much.

2. Desaturate Filter: When you first apply this filter, it will completely desaturate your image, i.e. make it B & W. Do not fear – you can change this. But first, a bit of explanation. Often the codec artifacts that appear will be more pronounced when there are highly saturated colors in the scene, i.e. bright reds, yellows, greens, or blues. So, you can cut down the choppiness of the look in these scenes by desaturating them a bit. When using this filter, choose a frame where there is a highly saturated color (if you have none, this filer probably wont be of much help) and add the effect. There is a slider on the effect that will allow you to change the saturation from black and white (none) to highly saturated colors which will probably bleed on your TV. Choose a happy medium – you do not want to desaturate too much, or else the movie will look drab and even depressing (if this is what you are going for, desaturate away!). Just pull back some of the bright colors if you have them, and you will find that some of the artifacts are a bit less imposing.

3. Deinterlace Filter: This filter depends on where you are showing the movie. If you are looking to show it on a TV (definitely not my recommendation if you capture YouTube videos, unless you are showing on a Apple TV), you cannot use this filter, as it will make the image go fun-house mirror on you. Also, if the image was not originally taken from a TV source, this filter probably will not help. So, all of this being said – this file I was working with was taken from a TV, and not deinterlaced, resulting in lines that appear and make the image look jagged when movement occurred. So, deinterlacing can help too.

All in all, the best solution to all of this if really to see if you can find the source video. Email the person who posted it on YouTube and ask if they can send you even a highly compressed version through email. A well compressed video, even highly so will be better than YouTube codec screen captures.

Perhaps there should be a rejoinder to the title of this post – “the bane of the video world.” I have been spending the last week attempting to transcode a number of videos in a variety of formats to high-quality Quicktime movies. One big obstacle began in the form of the size of the videos – most of them were very small, many 300 X 200. So to upscale these videos, yet still have quality that is reasonably free from YouTube-esque codec artifacts was a challenge – and was solved by encoding with the H.264 (x264) codec, multi-pass encoding, and not taking the size much past 640 X 480. This combination resulted in the best quality and the largest size possible.

The next obstacle was the formats themselves. Some were simply encoded in MPEG-2 or AVI formats, and proved relatively simple to transcode into a Quicktime MOV file. I found that with the MPEG-2 files, FFMpegX, a open-source encoding program tended to produce better results than the native compressor in Quicktime Pro as well as the Compressor application from the FCP Suite. Now for the reference in the title of my post. There were a series of movies in a DAT format – which is a kind of generic format that can store a variety of file types. When opened with QT Pro, the files played – but when exported into MOV format only video would be encoded (and only after an extremely long time at that!). All audio was absent. I pored over these files for days, searching high and low for a solution. Finally, I opened the inspector for the file in QT Pro and found that the file type was classified as “MPEG-1 Demuxed.” I typed this description into Google, and lo and behold, the first result was the solution to my problem.

Evidently, older video encoders would sometimes encode videos by “demuxing” short for “demultiplexing” – in short the video and audio were separated, but contained within the same container (hence the single file with the DAT extension). So when I opened the files in QT Pro for playback, QT Pro was able to find the audio files, but the program balked at trying to compress the separated files, resulting in a video-only MOV file. So, the solution came in the form of another open-source software program, titled MPEG Streamclip. This nifty program allows you to take the MPEG-1 video stream and AIFF audio stream out of the single DAT package, and separate them into two files – and upconverts the MPEG-1 into a MPEG-2 stream. Now, as long as the two files are named exactly the same thing, and located in the same place, QT Pro suddenly cooperates and find both the audio and video streams, and allows you to do a normal transcode to MOV files. Problem Solved.

Now, my final obstacles are two files that are in a folder which I have named “Problem Children” – because neither wants to cooperate with any of my decoding wiles. One is a file in a RMVB container – which is a kind of souped-up RealPlayer package for DVDs – this one I have some hope for – as I just need to get it out of the uncooperative packaging. However, there is a second that is somehow in a UNIX Executable file format – which I have little idea how a video would be contained in said package, and thus little idea as to how one might coax it out. Any counsel from seasoned parents of uncooperative video formats would be greatly appreciated.

EDIT: I figured out our impetuous files. Both proved a bit difficult – but I managed to figure them both out. The first, in the .rmvb format involved a cooperation between FFmpegX and RealPlayer. FFmpegX, the converter that I previously mentioned has the ability to convert regular RealPlayer files (found in the .rm format), but would not recognize .rmvb, which is a new format designed for the newest version of RealPlayer, and stand for “Real Media Variable Bitrate.” So, I had to make my way through a somewhat poor translation of a web page from Hong Kong to figure this out – so here is a simplified version:

1. Make sure both FFmpegX and the latest version of RealPlayer are both downloaded onto your computer. Search google for the specific websites and instructions dealing with FFmpegX installation.
2. In the finder, choose Macintosh HD –> Library –> Application Support –> ffmpegx –> reallib. There will be some files in this folder, but just leave this open for now.
3. In another finder window, go the Applications tab and find the RealPlayer icon. Control + click on the icon and choose “Show Package Contents.” This will open a new finder window.
4. In the new window click on Contents –> Frameworks –> HXClientKit.framework –> HelixPlugins –> Codecs. Now select all the files in this folder and while holding “option” drag them into the “reallib” folder you had open before. Be sure the files go into this folder and not one of the subfolders in it.
5. Now close all the finder windows, log off or restart your computer and open FFmpegX. Drag the .rmvb file into the interface. It will by defualt be converting the file to an AVI XviD format – IMPORTANT – you must leave it as this format for the initial conversion or it will not work. You can open the AVI file and recompress or transcode it as you please. The progress window that opens will not show a normal 0 -100 % progress indication, it will look like it is thinking – so just leave it. It will eventualy “Ding!” and you will have your nicely converted AVI file.

For the other file, the one for whatever reason contained in a “UNIX executable file” format – you need to use Terminal. As a word to the wise, save a backup of your file, and though you may be tempted, remain calm and do not mash keys into the terminal prompt – you might cause serious issues with your OS. So, here is the process for converting this type:

1. Open a new terminal script. You should see an interface that ends with a dollar sign. i.e. this computer looks like this: jarjar:~ ITG$
2. Begin by typing the word “file” and hitting the space bar once. Now drag the UNIX executable file into the terminal window and you should see the destination path for the file appear. Press enter.
3. This should, among other things give you the type of file that you have contained. Look for a recognizable extension – i.e. MPEG-1, MPEG-2, AVI, MPEG-4, etc.
4. Now the important part. You should see a similar interface as you previously did. This time, type “chmod -x” this is very important to get right. It is chmod(space)-x followed by another space. Drag the file into the window again. Hit enter.
5. Close terminal and the finder window, log off or restart your computer, and reopen the finder window, and find your file. It should now be blank white (it might still say something about a UNIX executable file, or “blank document”), now edit the name of your file and add the extension appropriate to that which you saw in the second step (i.e. if it is MPEG, give it a .mpg extension). Say yes to the warning box that will appear.
6. Now you should be able to open the file in QT Pro, although the file if in MPEG-1 format might be demuxed – see earlier in this post for directions on correcting this issue. Now the file should be able to be recompressed, etc. as you see fit.

Wow. Lots of fun right? Welcome to the past two weeks of my life! Feels good to solve problems though!

Working on the video blog for Mary Barr’s “Race in the American Community” class this summer, we ran into the problem of how to easily embed and play the students’ interviews within a web browser. One option was to use a standard Quicktime embed. Since the videos are directly coded as DivX .avis (actually Xvid with mpeg 2 audio – but I’ll spare you the esoteric details) right out of the camcorder, this would require mac users to download a third party codec suite (such as Perian) to play them. QT for Windows doesn’t really support playback of DivX .avis, so we would have to embed with Windows Media Player for PC users. Unfortunately, though, to make matters worse, WMP has embedding issues – it’s difficult to change the size of the player window or scale the movie properly. So, ultimately, we decided to just embed the videos as hyperlinks, where they would open up in a new window and either download or play in the user’s media player of choice. As a compromise, it was not a bad solution. But some videos were large, took a long time to progressively load before playing, and could not be integrated into the overall aesthetic feel of the blog.

It occurs to me (now that the class is over) that another solution would be to use the DivX web player, a relatively new cross-platform plugin that allows you to embed high quality .avi encodes on websites. I took one of the larger (152 MB) student-generated videos from Mary’s class and embeded it below using the web player. Let’s see how it turned out…

Creating a Post

This is just a little test from Screenflow, a pretty cool program that allows for screen capture on Macs. I made a tutorial for posting on WordPress, similar to my previous help pages. A major plus of this program is the ability to do “callouts,” which are when you see the little spotlight zoom in on the cursor. Also, as a cosmetic plus, I was also able to do the nifty reflection look.

Edit: Now it has my voiceover.

I recently completed another lynda.com tutorial (see the previous post on DVD Studio Pro for more details on lynda.com), this time dealing with Compressor 3, an application that comes with FCP, and can be used to compress video files into a variety of formats. All in all, Compressor is another application that comes with the FCP suite that is really optimized to be used with videos edited in FCP, and then to be imported into DVD SP for DVD creation. So, the same mantra as before applies here – a very powerful, versatile program but also a program that has a true learning curve. When opened, Compressor is rather imposing, having six windows, all serving different functions, and none of which seem to be a place into which one might bring a file.

After watching the tutorial, the process is demystified a bit, and the workflow begins to become more intuitive. However, unlike FCP and DVD SP, I found that the functions performed by Compressor are often fully duplicated in iMovie and iDVD. For example, Compressor and iDVD both allow one to select the quality of compression used in compressing the movie, and iDVD automatically detects the the size of your movie and optimizes compression for that. Also, in the “expert settings” pane of iMovie, one is able to compress movies into a wide variety of settings, one which arguably rivals that of Compressor. The settings for compression are fully customizable in both Compressor and iMovie, and iMovie arguably allows for less human error, with fewer entry-based fields and more drop-down menus.

One area where Compressor could be extremely useful is in its use of Droplets. A droplet is a template of customized settings for some operation (i.e. 90 minute DVD or High-quality Web Streaming) that allows you to simply drag and drop movie files, and then to have the settings applied to all of the files without ever having to open the application (Compressor does not actually do the compression; there is a batch application that runs along with Compressor that completes the process). The batch application opens, compresses the files, places them in a proper destination folder, and closes. This tool would be more expedient were one to be doing a large number of files with the same settings, and whereas iMovie would necessitate a series of clicks and dialog boxes for each file, a droplet would allow for a one-stop drag and drop process.

Compressor can also run “clusters” – or mutliple processors/cores if your computer should be so equipped. What this would mean is that if one had a computer with a quad-core processing system, rather than using only one core for the processor-intensive process of compression, they could use two, or even four (not recommended) cores. So, for some longer compressions, Compressor could call more then one processor cluster into active duty, and thus cut down on the time such an operation might normally take.

In conclusion, I think that Compressor is a great application, but is really most useful when used in tandem with the entire FCP suite. To be taking DV files from iMovie and trying to use Compressor would most likely just add an additional step that would result in a file that was exactly the same as could be created in iMovie or iDVD. I only wish I had that nifty droplet tool sometimes…

Ok, no more hijinx – thanks to Joe. An update on the Cold War materials; we figured out how to embed the clips from the Cold War documentary on Classes V2. This discovery has made many of us very joyous, as it will not require the time and effort necessary to build entirely new webpages, run comment scripts, etc. All of the various portions and vagaries of this arduous process can now be found at http://itghelp.commons.yale.edu. Without going into too much detail – Joe alterted me to an open source .flv player developed within the .swf format, allowing us to embed the player in a HTML page on the Classes V2 server and later embed the .flv file into that player. This development will now allow us to post these videos for the faculty member that requested them, have them only be available to the class, and further, have the videos available during the specific weeks that the faculty member would like them to appear. So, all in all, for now, the .flv hijinx has been de-jinxed. Thanks mostly to Joe, but with some further research, support and help from Pam.

In Attendance:

Pam Patterson – ITG

Ioannis Yessios – ITG

Matthew Regan – ITG

Andy Shimp – Engineering and Applied Science Library

Kalee Sprague – Library Technology Services

Purpose:

To consult with library technology specialists in regards for collaborative citation solutions, especially for faculty and students.

Notes:

The meetings started out with Pam explaining some of the obstacles ITG has run into recently with faculty creating citation lists and wanting to collaborate with students. One example given was Seth Fein who uses Endnote to cite works, as well as add metadata to his digital and video asset collection. A number of software tools are used to gather and add the pertinent information, including Extensis Portfolio, Adobe Bridge, QuickTime and Endnote. Seth also has a student modify citation lists on another computer. Because both use the local client of Endnote, it becomes tedious to update each list of the edits by exporting the files on one client and importing it on the other.

It was noted that Endnote has a web-based client, but the sharing is a bit unstable and the limit of citations is 10,000 which may not be ideal for users with larger citation lists, e.g., William Kelly.

Classes*v2 Citation Tool

Kalee demonstrated the new Classes*v2 citation tool. Developed with funding from Mellon Grant

Features:

• It is found in the Resources section
• A new list can be created by selecting “Add Citation List” from the Add drop-down
• Can choose a search category
• Allows to search databases from Yale and Google Scholar
• Can import RIS files from either Endnote or RefWorks
• Can configure permissions for user access
• Has annotation capability
• Able to modify list (add/remove citations)
• Need to use VPN to access Yale restricted resources
• A completed list can be viewed either in Citation View or Title View

RefWorks

RefWorks is a web-based citation list tool that the University has licenses for Yale affiliates.

Features:

• Plug-in for MS Word
• Has RefShare feature to give access to a group
• Can import/export RIS files
• Has a RSS feed

RefWorks may be a solution for both Seth Fein and William Kelly. Pam created an account for ITG and Yianni was going to import William Kelly’s list of 20,000 citations to test stability/capability/etc.

Imported Seth’s citation list of 712 without issue and tested export formats.

Questions/Concerns/Testing:

• Stability of RefWorks for large citation lists
• Importing RIS files back to Endnote; overwriting, loss of data, etc.
• Exporting data from RefWorks into a format that will link with Portfolio
• MS Word Add-in

Recently, with the advent of a standalone .flv encoder and a better codec for encoding flash video (On2 VP6), we at ITG decided to try and take some videos on the Cold War that currently reside in .rm and .mov containers and move them to the .flv format. Firstly, .flv allows for embedding within a webpage so that the file is not able to be easily downloaded by users (.rm and .mov files can also be made to run in a “kiosk” mode, but it often proves somewhat difficult). Secondly, the files created with .flv encoding are often a good deal smaller than those in the .mov or .rm format. However, .flv files take a very long time to encode (often the true length of the clip or 1.5x that amount depending on the source format; i.e. dv to flv is truly a task, one is almost better off trading off on the reduced quality of going dv to .mov and then to .flv!). Also, we ran into the issue of where and how to get these .flv files from the encoder to a website that could be accessed by students. We made an attempt at embedding the files in Classes V2, using the internal FCK editor. The issue soon arose with the FCK editor only allowing .swf (another flash format) files to be embedded. As the process for encoding a .swf files is even longer, and would result in unmanageably large files – we made a few attempts at using DreamWeaver to give us code with the .flv file embedded, but to no avail. In the end we decided that the process of getting these videos up for students would take a bit of web developing and some good old-fashioned manual editing so as to allow the files to be available week by week. So, in the end, the .flv format has many advantages – but some kinks need to be worked out for it to be a viable format to use in a variety of settings, as evidenced by this posting.