ffmpeg

July26

Loads of pain converting simulation output (ppm and TIF) time series into a movie format that a journal likes has led me to start this page. Heres a list of standard ffmpeg (Mac OSX 10.12.3, Terminal) tools used on Terminal with standard tools. You might need to install some of them if you are on some flavour of Linux (or not).

Caveat: I’m using a MAC OSX 10.12.5 (Sierra) with Xcode and multiple gnu developer tools installed using Mac Ports.

1) Convert .avi files to MAC readable (not just VLC) .mp4 files:

ffmpeg.exe -i %d.png -f mp4 -vcodec libx264 -pix_fmt yuv420p test.mp4

The .png file could be an input .avi file (in which case drop %d.png for your file series)

Convert to MOV using QuickTime->Export.

2) Convert pre-existing movie files in Quicktime .mov format to uncompressed AVI or even MAC compatible .mov

>>ffmpeg -i terasaki.mov -vcodec rawvideo -y terasaki.avi

I recently needed this to make Supplementary Materials files ImageJ readable and neeed UNCOMPRESSED avi files (all compression codecs seemed to trip up ImageJ). The output looks like this:

ffmpeg version 1.2.1 Copyright (c) 2000-2013 the FFmpeg developers
built on Jun 12 2013 13:46:16 with Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
configuration: –prefix=/opt/local –enable-swscale –enable-avfilter –enable-libmp3lame –enable-libvorbis –enable-libopus –enable-libtheora –enable-libschroedinger –enable-libopenjpeg –enable-libmodplug –enable-libvpx –enable-libspeex –enable-libass –enable-libbluray –enable-gnutls –enable-libfreetype –mandir=/opt/local/share/man –enable-shared –enable-pthreads –cc=/usr/bin/clang –arch=x86_64 –enable-yasm –enable-gpl –enable-postproc –enable-libx264 –enable-libxvid
libavutil      52. 18.100 / 52. 18.100
libavcodec     54. 92.100 / 54. 92.100
libavformat    54. 63.104 / 54. 63.104
libavdevice    54.  3.103 / 54.  3.103
libavfilter     3. 42.103 /  3. 42.103
libswscale      2.  2.100 /  2.  2.100
libswresample   0. 17.102 /  0. 17.102
libpostproc    52.  2.100 / 52.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘terasaki-mk0780641002.mov’:
Metadata:
major_brand     : qt
minor_version   : 0
compatible_brands: qt
creation_time   : 2017-06-19 06:11:05
encoder         : Mac OS X v? (AVF 1046.9.12, CM 1731.15.207, x86_64)
encoder-eng     : Mac OS X v? (AVF 1046.9.12, CM 1731.15.207, x86_64)
Duration: 00:00:04.67, start: 0.000000, bitrate: 926 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0×31637661), yuv420p, 370×220 [SAR 1:1 DAR 37:22], 924 kb/s, 6 fps, 6 tbr, 60 tbn, 120 tbc
Metadata:
creation_time   : 2017-06-19 06:11:05
handler_name    : Core Media Data Handler
Output #0, avi, to ‘terasaki-mk0780641002.avi’:
Metadata:
major_brand     : qt
minor_version   : 0
compatible_brands: qt
encoder-eng     : Mac OS X v? (AVF 1046.9.12, CM 1731.15.207, x86_64)
ISFT            : Lavf54.63.104
Stream #0:0(eng): Video: rawvideo (I420 / 0×30323449), yuv420p, 370×220 [SAR 1:1 DAR 37:22], q=2-31, 200 kb/s, 6 tbn, 6 tbc
Metadata:
creation_time   : 2017-06-19 06:11:05
handler_name    : Core Media Data Handler
Stream mapping:
Stream #0:0 -> #0:0 (h264 -> rawvideo)
Press [q] to stop, [?] for help
frame=   28 fps=0.0 q=0.0 Lsize=    3345kB time=00:00:04.66 bitrate=5871.8kbits/s
video:3339kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.188078%

For the mac OSX compatible movie files (quicktime playable), a small conversion with decocer fix:

>>ffmpeg -i videoS2.mov -pix_fmt yuv420p videoS2-v3.mov

Falls under the entry Encodingfordumbplayers


References:

1) Stackoverflow: http://stackoverflow.com/questions/14430593/encoding-a-readable-movie-by-quicktime-using-ffmpeg

2) FFMPEG User-List http://www.ffmpeg-archive.org/Uncompressed-AVI-to-Uncompressed-Quicktime-td941357.html

3) https://trac.ffmpeg.org/wiki/Encode/H.264#Encodingfordumbplayers

Latex: Better Scientific Documentation

May8

Bibtex

Otherwise my favourite means of maintaining bibliographies. A small list of useful bibstyle files that I needed to modify

Journal Citation type Reference type .bst file
Proceedings of the Indian National Science Academy (PINSA) (Athale and Chaudhari 2010)

For multiple references (Athale and Chaudhari 2010, Khetan and Athale 2015)

Chaphalkar A R, Jain K, Gangan M S and Athale C A (2016) Automated Multi-Peak Tracking Kymography (AMTraK): A Tool to Quantify Sub-Cellular Dynamics with Sub-Pixel Accuracy PLoS ONE 11 e0167620 procinsa.bst

For equations, consistent formatting and generally a good input output relationship (you get out what you put in plus some processing, predictably and reproducibly), LaTex has been used for a long time. Here are some notes on my usage of the tool and other links that might be useful for those interested in using it in future. The first stop is usually CTAN.

Latex on Mac OSX

Latex engines come integrated with the Mac OSX but it helps to download the comprehensive LaTex package (Tex Live)  before proceeding. I have been happily using TexShop as a LaTex composer tool.

Devanagari and Indic Fonts

Seems like using

\usepackage{devanagari}

and compiling some sort of phonetic hindi gets you the characters but not quite the whole thing. Some more searching led me to the understanding that XeTex is needed on top of the TexLive I am using to compile Latex (on the Mac OSX).

Bengaali on latex but the tricks here should work for Devanagari too.

By Jupyter

September19

The Romans are under attack. "By Jupiter"

First steps in installing Jupyter on my Mac OSX 10.12.6 Sierra

  • Getting the Xcode command line tools
  • Install MacPorts https://www.macports.org
  • Install Python 3.5 >>sudo port install python35
  • Install PIP from their site using a curl script to get the most updated version https://pip.pypa.io/en/latest/installing/
  • If you got to this stage then pip should allow Jupiter to be installed. Not yet there.

Mac for scientific computing

December11

Some VERY USEFUL programming tips from Jim Germain, U-Utah. Particularly useful for bare-bones scientific coding.


Some steps in installing MAC PORTs on a MAC BOOK PRO (OSX 10.12.1) Sierra or how to turbocharge your Mac (terminal-ly speaking):

1. Ensure your root password is turned on

2. Refer to the manual of Mac ports

3.  Download the pkg file with the Mac Posts for Sierra


Octave packages can be installed based on this set of instructions but briefly:

1. download the package in *.tar.gz format

2. cd to the directory and run Octave

3. Run the command $pkg install mypack.tar.gz
If no errors show up, that’s great. Usually some dependencies might be missing.


Snippets of things that didn’t work and workarounds for MAC OSX intel systems.

  • Octave not installing on sudo ports install octave. Solution: Install a flavour
port install octave +accelerate +gcc45
  • In a Linux box if you need to know if the system is 32bit or 64bit, the command is:

uname -a

which should give X86_64 or X86_32.

Alternatively:

getconf LONG_BIT

Post-publication review and PLOS’ experiment with the Synthetic Biology Collection

October28

An inducible luxI system (iptg) to produce the AHL above a threshold Pt. Kadam et al. (2016)

The iGEM 2015 synthetic biology contest was an important one for us. It marked our first attempt at putting together a project from IISER Pune. But beyond the novelty for us, many things were different this time around (#igem2015). First off, no preliminary or elimination rounds.

Secondly, we (yes, some self-backpatting here) organized an India Meetup in the run-up to the Jamboree. And third, and interestingly, the journal PLOS One (Public Library of Science) decided to use this as an opportunity to launch the PLOS iGEM collection, as a sort of meta-list, connected to iGEM. They decided to also go the radical way- with post-publication review. Time will tell how this latter experiment works out. And naturally our team’s efforts are there. With a lot of hard work put in by Snehal Kadam well after the contest and some griding-the-article together by mining long-forgotten (1 year ago!) protocol books, and some frantic emailing and interviewing, we managed to pull it off. You can read it here “Mycobacterium Revelio: Characterizing and Modeling Genetic Circuit Components towards a Bacterial Detection Tool”. The first 10 authors are BS-MS undergraduate students. Manasi and Neha are PhD students.

Google scripting

October21

Survival VI (visual editor)

September27

A simple set of commands you MUST know to use VI (visual editor), the editor that comes bundled with most UNIX/Linux flavoured systems.

Basically it has 2 modes- command and edit.

Usually you will want to edit something. So you press the letter “i”. This will show in the lower left corner as “INSERT” mode. So you know what state it is in. And you can delete, type text etc. If you typed the stuff well and want to save the changes made, press “:x”, else “:q!”. Usually “:” escapes from the edit mode, and brings up a cursor in the bottom left corner of your screen. Then x, q, or q! will determine the mode of action.

In the command mode you can yank (yy = copy) or paste (p = paste) text.

More detailed help:

coloradostate

New Bibliography Formats in Mendeley

June5

Modifying Mendeley

Using mendeley the citation manager tool, I needed to expand the limited formats available. To create your own citation style:

1) Citation Style Language (CSL) file with the right specifications. This is a form of XML. Use a regular text editor and store is as *.csl.
2) Where to store it? * Windows Vista/Windows 7: %LOCALAPPDATA%\Mendeley Ltd.\Mendeley Desktop\
* Windows XP: C:\Documents and Settings\«Your Name»\Local Settings\Application Data\Mendeley Ltd\Mendeley Desktop\
Note that you might need to enable the “Show hidden files and folders” setting in Windows Explorer’s folder options to be able to navigate to the Mendeley data folder.
* Linux: ~/.local/share/data/Mendeley Ltd./Mendeley Desktop/
* MacOS: Macintosh HD -> /Users/«Your Name»/Library/Application Support/Mendeley Desktop/
3) Restart mendeley. It should show up in the styles installed.
4) Alternatively if Mendeley has posted the .csl file online get it from Zotero.

posted under Blog, Scientific computing | Comments Off