2014-07-13

raw image format, ffmpeg, and raspivid

The SwView.org node 165: RGB565 To PNG/JPEG showed up at the top of the results listing for a search on RGB565, an image data format produced by a camera. We are most interested in capturing an manipulating said data, so we want to know what format it's in, and this RGB565 is used to refer to the format throughout the files which we currently access.


if image.raw contains data for multiple frames (e.g. of a video), you can use "%d" and ffmpeg will automatically name the additional files generated: 
ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 1024x768 -i movie.raw -f image2 -vcodec png image%d.png

Instead of a file, If you have an rgb565 stream, you may attempt to direct the data to stdin of ffmpeg and accomplish the same. In such a case, instead of "-i image.raw", you can use "-i -":
firstCommand | ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 1024x768 -i - -f image2 -vcodec png image%d.png

Note that raspivid can produce raw video and place it on stdout.

2014-07-12

RasPi HDMI Init /boot/config.txt working changes

These are the working settings from /boot/config.txt on the Raspberry Pi Raspbian install -- only the editted lines are shown - all the other settings were left at the defaults.

TODO: make a diff

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
disable_overscan=0

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
overscan_left=-192
overscan_right=128
overscan_top=16
#overscan_bottom=-6

# uncomment to force a console size. By default it will be display's size minus
# overscan.
framebuffer_width=1280
framebuffer_height=960

2014-07-11

Tweaking Rapsberry Pi HDMI

Tweaking Rapsberry Pi HDMI

fixed monitor size problem for acer panel + raspberry pi

Found several interesting things while examining the drive image(s) for the 8-gig Raspbian image that became inoperable on the Pi during intensive installation and removal of build packages...

  1. The Raspbian Linux install image is actuall more than one partition. When the SD card is plugged into a USB SD card reader connected to the T42 system, two partitions are shown [and optionally auto-mounted] for the device. These show up here under Mint / Debian ~ Ubunbtu as

        /dev/sdb1
        /dev/sdb2 
    

    which are mounted at

        /media/boot
        /media/fc254b57-8fff-4f96-9609-ea202d871acf
    
    respectively.

  2. Not sure why /dev/sdb2 is not mounted on /media/sdb2 as one might expect...

    Further investigation includes checking the mount data from within the pi acconut on the Raspbian install on the Raspberry Pi device itself.

    This shows that the root file system of the Pi is mounted as

        /dev/root  -> /
    

    Since there is no way to identify hard disc partitions from that mount listing, we try

        ls -lh /dev/root
    

    which returns

        lrwxrwxrwx 1 root root 9 Jul 11 08:10 /dev/root -> mmcblk0p2
    

    so we try

        ls -lh /dev/mmcblk0*
    

    which gives us

        brw-rw---T 1 root floppy 179, 0 Jan  1  1970 /dev/mmcblk0
        brw-rw---T 1 root floppy 179, 1 Jan  1  1970 /dev/mmcblk0p1
        brw-rw---T 1 root floppy 179, 2 Jul 11 08:10 /dev/mmcblk0p2
    

    ... furthermore, looking in the /boot directory of the filesystem mounted on / we see that there is nothing there -- it is not the location of the filesystem automounted from /dev/sdb1 on /media/boot on the T42 system.

    So this:

       sudo mount /dev/mmcblk0p1 /boot
    

    and

        ls -lh /boot total 9.5M
        -rwxr-xr-x 1 root root  18K Jul 10 02:48 bootcode.bin
        -rwxr-xr-x 1 root root  120 Jun 20 06:51 cmdline.txt
        -rwxr-xr-x 1 root root 1.2K Jul 11 08:10 config.txt
        -rwxr-xr-x 1 root root 2.1K Jul 10 02:48 fixup_cd.dat
        -rwxr-xr-x 1 root root 5.8 K Jul 10 02:48 fixup.dat
        -rwxr-xr-x 1 root root 8.7K Jul 10 02:48 fixup_x.dat
        -rwxr-xr-x 1 root root  137 Jun 20 10:33 issue.txt
        -rwxr-xr-x 1 root root 3.1M Jul 10 02:49 kernel.img
        -rwxr-xr-x 1 root root  19K Sep 25  2013 LICENSE.oracle
        -rwxr-xr-x 1 root root 503K Jul 10 02:48 start_cd.elf
        -rwxr-xr-x 1 root root 2.5M Jul 10 02:48 start.elf
        -rwxr-xr-x 1 root root 3.4M Jul 10 02:48 start_x.elf
    

    Which leads us to the next fascinating discovery:

  3. HDMI Boot-time Cofiguration Parameters

    cat /boot/config.txt 
    

    ... and find ...

        # uncomment if you get no picture on HDMI for a
        # default "safe" mode 
        hdml_safe=1
    
        # uncomment this if your display has a black border 
        # of unused pixels visible and your display can 
        # output without overscan 
        disable_overscan=1 
        # uncomment the following to adjust overscan. Use
        # positive numbers if console goes off screen, and 
        # negative if there is too much border 
        #overscan_left=16 
        #overscan_right=16 
        #overscan_top=16 
        #overscan_bottom=16 
    
        # uncomment to force a console size. By default 
        # it will be display's  size minus overscan. 
        framebuffer_width=1280 
        framebuffer_height=960 
    
        # uncomment if hdmi display is not detected and 
        # composite  is being output 
        #hdmi_force_hotplug=1 
    
        # uncomment to force a specific HDMI mode (this will 
        # force VGA) 
        #hdmi_group=1
        #hdmi_mode=1
    
        # uncomment to force a HDMI mode rather than DVI. 
        # This can make audio work
        # in DMT (computer monitor) modes
        #hdmi_drive=2
    
        # uncomment to increase signal to HDMI, if you 
        # have interference, blanking, or no display
        #config_hdmi_boost=4
    
        # uncomment for composite PAL
        #sdtv_mode=2
    
        # uncomment to overclock the arm. 700 MHz is the
        # default.
        #ar_freq=800
    
        # for more options see http://elinux.org/RPi_config.txt
        gpu_mem=128
        start_x=1
    

    Interesting - this config.txt file looks like it might have something to do with the raspi-config utility that's used to change many of the settings that are listed in the file.

    Even more interestingly, I don't remember seeing these setting in the curses interface raspi-config always presents. So. No idea what this stuff is, but it looks like these lines

        # uncomment if you get no picture on HDMI for a default "safe" mode
        hdmi_safe=1
    
        # uncomment this if your display has a black border of unused pixels
        # visible and your display can output without overscan
        disable_overscan=1
    
        # uncomment the following to adjust overscan. Use positive numbers if 
        # console goes off screen, and negative if there is too much border
        #overscan_left=16
        #overscan_right=16
        #overscan_top=16
        #overscan_bottom=16
    
        # uncomment to force a console size. By default it will be display's 
        # size minus overscan.
        framebuffer_width=1280
        framebuffer_height=960
    
        # uncomment if hdmi display is not detected and composite 
        # is being output
        #hdmi_force_hotplug=1
    
        # uncomment to force a specific HDMI mode (this will 
        # force VGA)
        #hdmi_group=1
        #hdmi_mode=1
    
        # uncomment to force a HDMI mode rather than DVI. This can 
        # make audio work in DMT (computer monitor) modes
        #hdmi_drive=2
    
        # uncomment to increase signal to HDMI, if you have 
        # interference, blanking, or no display 
        #config_hdmi_boost=4
    

    ... may well have to do with the HDMI Video Sub-System Startup operations - not sure why this would be on a seperate partion, and not sure what the other files in this directory are, yet, but uncommenting the hdmi_safe line, the disable_overscan line, changing the framebuffer_width and framebuffer_height lines (uncomment and edit to have the values shown above) fixed the problem we were having of the Rapsberry Pi HDMI output locking itself to 1600x900 on acer 20" flat panel - the panel was intializing itself to 1280x960 - now it running with the X display nicely centered on the screen, but the panel's bezel-buttun MENU says that the display is set to 640x480.

    ... this after rebooting the Pi with settings as shown above; am guessing that the hdmi_safe=1 line may be responsible am checking the display dimensions again, commenting out the "safe" line again, and rebooting.

    So. After several hours messing about with these settings the best I can do right now is this:

        #hdmi_safe=1
        #disable_overscan=1
        overscan_left=-192
        overscan_right=96
        overscan_top=16
        overscan_bottom=0
        framebuffer_width=1280
        framebuffer_height=960
        #hdmi_force_hotplug=1
        #hdmi_group=1
        #hdmi_mode=1
        #hdmi_drive=2
        #config_hdmi_boost=4
    

    This gets the entire display on the screen (or at least so it appears thus far) but there is still a wide black border across the bottom of the screen - the overscan value can cause the console to write off the bottom edge of its display, but it does not reduce the size of that border area.

  4. The aspect ratio of the screen seems whack - 1280x960 - and there are obviously lines of pixels across hte bottom of the display that are not getting lit up - it looks like about 64 of them. This is as far as I'm prepared to go with this right now, since the immediate problem of the Pi drawing most of it's display off the right edge of the screen now seems to be solved.

2014-07-09

Compiz Fusion Tray Icon (...or: Getting Rid of MATE)

Compiz Fusion Tray Icon

or: "Getting Rid of MATE"

I have been running the MATE desktop which seems to be instlaled by default with Linux Mint.

I am finding that this Gnome2-derived MATE is bulky and slow, providing little in the way of advantages for the user, and much in the way of waiting and annoyance.

So, on impulse, and while installing FVWM and Xfwm4, I installed the "Compiz Fusion Tray Icon" package which was offered in the Aptitude listings.

  1. Install Compiz Fusion Tray Icon

I was doing this while running in "recovery mode", so the next thing after completing the installations and exting Aptitude, was to continue the boot-up.

On boot, the MATE desktop came up. This was unexpected. Am not sure what happened to the login screens and the option to choose a desktop environment...

Within the MATE desktop, the newly installed Compiz package is in the menu, but is not running -- again this is somewhat unexpected.

  1. From the Mint/MATE/Gnome2 panel ("Start") menu, under Adminitration, select the Compiz Fusion Icon menu item. This will start Compiz Fusion Tray Icon in the Notification area of the panel.

  2. Right-click the fusion icon in the tool bar and choose Select Window Manager from the context menu.

  3. The Select Window Manager menu option displays a sub-menu showing a list window managers (e.g. Compiz || Marco || FVWM || Metacity || Xfwm4 ). This is the list of installed window managers which are available for use, here.


  • using the markdown everywhere Chrome app under Linux
  • Desktop is composited Xfwm4 using Compiz
  • Chrome apps install to panel menu under Other

2014-07-08

How to Build Raspbian Source Packages


— How to Build Raspbian Source Packages

:Document created 2014-07-07 05:44:51 by pdwilso@gmail.com


How to Build Raspbian Source Packages

Overview

Using the instructions found in HowTo Build a Package from Source the Smart Way ./Build-Pkg-Smart-Way.pdf.tar.gz as a guide, we have performed the following steps:

Setup

1. Freshly installed Raspbian image on SD card
 
2. Edit /etc/apt/sources.list
sudo vi /etc/apt/sources.list
... to add the line ...
deb-src http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi

... then save the file.

The resulting sources.list file on the Pi looked like this:
deb http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi
deb-src http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi
3. Run sudo apt-get update
 
4. Run sudo apt-get build-deps <packagename>
 
Note that these steps outlined above are mostly analogous to the steps outlined in the PDF file, except that we are concerning ourselves exclusively with what we can get from the raspbian repository. This is what we want — I.E. we do NOT want to have to seek upstream from Raspibian into the Debian package repositories - the only ones were interested in are those that have already been built for our platform/distro, and those that will be built for our platform / distro.
 
This approach presumes that all the Debian APT package manipulation and management commands will work using only Raspbian packages and repositories.
I am not going to detail the original steps given in the PDF here in the interest of readability of this document, however, at this point in the guide document, the author states clearly:
 
"Those are the last commands you should run as root before installing the DEB. All of the build commands should be run as normal user."
 
When we say "at this point in the procedure" we mean:
  • Have added a source package repository to sources.list
  • Have run 'sudo apt-get update' against the new repositories.
  • Have installed build dependencies for package(s) to be built.
Also note that everything up to this point has been package-agnostic — that is,
  • these steps need to be performed for all source package builds.
  • The edit of the sources.list only needs to be done when a new source package repository is added.
  • The apt-get build-deps operation should be performed each time a new source package is to be used — remember packages can be updated, and there are perhaps other configuration changes which could cause the package build dependencies to change.
Once the build dependencies are installed by root, the action shifts to the user account.

{* In this case, we're on a Vanilla Raspbian Raspberry Pi, so the user is 'pi' *}

Get the Source Code Package

As ordinary user we do the following
apt-get source <packagename>
Issue this command while the Current Working Directory (CWD) is set to the parent directory of the source tree that will be installed for the package we're going to build.
 
Note that <packagename> is the Debian APT package name of the package you want to build. I.E. NOT the file name. In this case, we're using package name 'xscreensaver' to install the screen hack sources. The version number of the package is appended to the package name when the package is retrieved from the repository.

Dependency on devscripts Pacakge

The next operation listed in the guide document did not work on the first try, and investigation of that problem uncovered a dependency that was not listed — the devscripts package, which contains the deb utilities, including debchange, the one suggested by the guide as a way to change the version number of the source package. If one wants to do this — and the rationale behind is not yet entirely clear to me, I admit — then you will need to have the devscripts package installed.
sudo apt-get install devscripts

Using debchange

Going to cover some specifics of using the debchange utility (from the devscripts package mentioned above) to change the Version of the installed source package; doing this here because it was not covered in the document we're using as a guide, and it's obscure enough information that it should probably be covered in a how-to of this nature...
 
The use of the command given in the Debian PDF document is
debchange -b -v N.N.Nc-N~bpoNN+N
... where the string N.N.Nc-N~bpoNN+N is the version to assign to the package going forward. Furthermore, the version string assigned by the example includes the version of the source package installed (N.N.Nc-N) conjoined to 'bpo' — which is a TLA for 'BackPOrt' — followed by what one might presume is a backport version number conformant to some backport numbering scheme ... by a '~' character.
 
Note that in the above paragraph, 'N' indicates one or more digits, a '.' indicates a literal period ('.'), and 'c' indicates an alpha-numeric character. These together make up the version string or version number string.
 
So. Reading the manual page for debchange (installed with devscripts package, above) we find that indeed this utility is — or can be — used to adjust the version numbering of source packages...
 
The manual page also shows the reason the name of the package is not mentioned in the command line to change the version number (something I'd been wondering about since I first read the how-to begin with). The reason is simple, and given in the second sentence of the first paragraph of the manual page:
 
debchange or its alias dch will add a new comment line to the Debian changelog in the current source tree. This command must be run from within that tree.
 
So - the only other argument given on the debchange command line is -b for which the manual page tells us:
 
--force-bad-version, -b
   Force a version number to be less than the current one
   (e.g., when backporting).

Choosing a New Version String

So it's the "backporting" process I don't understand, it seems. or the versioning of it. Backporting as a topic is beyond the scope of this document and I'm not going to try to cover it here. Or even use the word.
 
For instance, I can see how the author of the Debian How-To set the version number to an older, "bad" or 'backported' version, I'm just not clear on why the version was handled in this way....? I think what I may be looking for is a way to make my own, local branch of the code. That would give me a local package to maintain, and a way to maintain and document changes, additions, and deletions to and from the code.
 
Based on remarks made in the serverfault.com thread How should I create a local fork of a Debian Package?, we will proceed with the idea that using debchange to induce a new version number of <existingverstr>+0x0000.00.00 or some such will be sufficient to protect the system package system as well as provide some revision control for changes to the source package(s).

Look into using Git to manage Source Package sources?

Answers in this thread <http://serverfault.com/questions/65858/how-should-i-create-a-local-fork-of-a-debian-package> suggest using git to fork a local branch from a public repository.

2014-07-07 04:20:55
pdwilso@gmail.com

    There is more to this - we haven't run the Debian Package Build Tools yet.**

Building the Source Package

The next step is (still as ordinary user, and within the Source Package directory):
dpkg-buildpackage -rfakeroot -us -uc

This command did not work, returning a failed dependency 'unable to build config.status' or something like that.

Continuing build process using make.

Building using Make

A number of warning regarding set but unused variables.

Also got:

/usr/include/gtk-2.0/gtk/gtkitemfactory.h:47:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]

{* 2014-07-07 19:38:03 — material moved here from :Calendar:2014:07:07 *}

Ultimate Failure of Guiding Document

2014-07-07 18:59:07 — Directly quoting The Guiding Document below; this is the point at which the process ceased to work on the Raspberry Pi — the command shown below failed with minimal error reporting, and no real clue about how to proceed.

From The Guiding Document:

"Next, build the DEB:
dpkg-buildpackage -rfakeroot -us -uc
That last command may take a minute or an hour or three hours. It all depends on the size of the package and your own hardware."
 
That just didn't work. The dpkg-buildpackage operation ran for quite a long time, then failed without waring, complaining about some config.status file which apparently may or may not have existed in some file system at some time etc.
 
I believe the goal of this command is to build a the package binaries from the source code, then build a deb package from the binaries, which can be installed later. {*  2014-07-07 19:26:25  * NOTE: we have already successfully built the package using ./configure ; make from the command line. *}
 
The Guiding Document goes on to assume that there were no problems at all with their arcane command:
"Once the dpkg-buildpackage command finishes, you'll find the DEB(s) in your /home/XXXX/my_build/ directory and you can install them (as root) with:

cd /home/XXXX/my_build/

dpkg -i <packagename>_<version>_<architecture>.deb

It doesn't get much simpler than that."

.... well, it may not get much simpler than that, but it doesn't get much more dysfunctional than that, either, and we should all at this point know better than to sacrifice functionality to simplicity. That is, it may be dead simple, but that's not an advantage if it doesn't do anything — or worse, if it fails without explanation, as this did.

If one continues reading in The Guiding Document (which we almost didn't — should fix some stuff, there, you Debian Forum authors, you) we find that there is — directly subsequent to the above non-sequitur in The Guiding Document — a detailed and pointed discussion of a tool called "dh-make" which may run some other utility or script which may set some variable or perhaps create or alter some files which may or may not affect the build — it appears they are — tacitly — pointing the way to a solution to the unacknowledged problem which many users had to have experienced trying to follow The Guiding Document's earlier, flawed instructions.

We shall see.


2014-07-07 19:23:31


  • dpkg-buildpkg docs
  • dh_make docs

Guiding Document Fail

Okay, this is where the guiding document completely fails to address the situation — there is no 'dh_make' command on the Rapsberry Pi vanilla Raspbian Latest — not even with devscripts already installed. We will do a system update for awhile, and come back to this later.

The most recent best clue is the manual page of something called "debhelper(1)" — apparently dh indicates 'debhelper' — and we can see that the debhelper command has command line options to build packages. More later.

dpkg-buildpackage.1.txt

According to the dpkg-buildpackage manual page, the parameters given for the failing dpkg-packagebuild command ( -rfakeroot -us -uc) have the following meanings:

-rfakeroot
-us
UnSigned; Do not gpg-sign the package build
-uc
Do not gpg-sign the .changes file
 
None of this really explains why the dpkg-buildpackage operation failed, and since the sources are already configured and building correctly using make we will leave the debugging of the the debian howto until later...
 
 

2014-07-06

Raspberry Metalflake

 rpiplayground: Hacking the GPU for Fun and Profit (Part 1 of 4)
Fun for those rainy days... Says the author: 
"This series of posts will walk through the GPU architecture by designing and optimizing a non-trivial algorithm (parallel SHA-256) to run on the GPU.  Because this is a relatively advanced topic, some background knowledge must be assumed. [ … ] You should have some understanding of GPGPU concepts (Google can help) and you should read the Broadcom BCM2835 GPU Documentation. I’ll describe most Raspberry Pi GPU specific concepts but in case I miss some, the documentation is your friend."
[ed note: we will try to cover some of those assumed bits of background knowledge here as we find out what they are. kthx]
SHA-256 implementation on QPUs
Raspberry Pi Forum thread where the rpiplayground (see link above) blog showed up.
OpenVG™ – The Standard for Vector Graphics Acceleration
From the page:
"… royalty-free, cross-platform API that provides a low-level hardware acceleration interface for vector graphics libraries such as Flash and SVG. OpenVG is targeted primarily at handheld devices that require portable acceleration of high-quality vector graphics for compelling user interfaces and text on small screen devices - while enabling hardware acceleration to provide fluidly interactive performance at very low power levels."
hermanhermitage/videocoreiv
Tools and information for the Broadcom VideoCore IV (RaspberryPi)

Simon's accelerated X development thread
This thread at the Raspberry Pi Forum leads to quite a few interesting links – starting with the low level virtual machine code repository:  simonjhall/llvm_qpu – Low Level Virtual Machine
GitHub repository for the LLVM sources – this is a frequently updated mirror of the official site … note that this is not anything to do with the java virtual machines –
"This directory and its subdirectories contain source code for the Low Level Virtual Machine, a toolkit for the construction of highly optimized compilers, optimizers, and runtime environments."
other simonjhall repositories on github: 
My Robot Blog:  GPU Accelerated Camera Processing On The Raspberry Pi
am very actively trying to follow this work currently

2014-07-01

freecode.com goes static

freecode.com is moving to a static site due to lack of use. Following are some links to content from the site which may be of interest or use: 
dumpmpeg project
— "a simple program that allows you to dump frames from mpeg movies. Future features may include dumping of the audio as well. dumpmpeg uses Loki's SMPEG library for decoding."
PHP OAuth API
"PHP OAuth API authorizes user access using the OAuth protocol. It abstracts OAuth 1.0, 1.0a, and 2.0 in the same class. It provides built-in support for popular OAuth servers: 37Signals, Amazon, Buffer, Bitbucket, Box.net, Dailymotion, Discogs, Disqus, Dropbox with OAuth 1.0 and 2.0, Etsy, Eventful, Facebook, Fitbit, Flickr, Foursquare, github, Google with OAuth 1.0a and OAuth 2.0, Instagram, LinkedIn, Microsoft, Rdio, Reddit, RightSignature, Salesforce, Scoop.it, StockTwits, SurveyMonkey, Tumblr, Twitter, VK, Withings, Xero, XING, and Yahoo!. Every other OAuth server is supported by setting an endpoint URLs and other parameters using specific class variables or an external JSON configuration file. The class can also send requests to an API using the previously-obtained OAuth access token. For servers which support offline access, the class can renew expired tokens automatically using refresh tokens."
Possum
"Possum is a point of sales (POS) system for restaurants, bars, and brasseries. It is intended for use with a ticket printer and a touch screen. It can be used to provide billing and presents different statistics such as sales, VAT amount, total payments by payment type, settings, average ticket per table, etc. These statistics are calculated per day and per month. It can also manage music playlists."
OpenSimulator
"OpenSimulator is a multi-platform, multi-user 3D distributed virtual environment platform. Out of the box, it can be used to simulate virtual environments similar to that of Second Life. These can be used as social virtual worlds or for specific applications such as education, training, and visualization. Access is via the regular Second Life open-source viewer or via third-party clients. There are a number of private and public deployments of OpenSimulator, including OSgrid, which has over 8000 regions hosted by independent individuals and organizations spread over the Internet."
Horde Application Framework
"The Horde Application Framework is a modular, general-purpose Web application framework. It provides an extensive array of classes that are targeted at the common problems and tasks involved in developing modern Web applications."
See also the profile for user chuck for more plugin-type, horde-friendly apps.