| Greg's computer video recorder project |
|
||||||||
|
|
|
|
|||||||||||||
This page describes the fun I have had setting up a computer as a video recorder, covering the time period April to December 2006. I also have older information which hopefully won't be needed except to document the pain, and the current log. See the main page for an overview.
Inevitably, I make mistakes. As I gain experience, I realize these mistakes; rather than just correcting the text, I prefer to leave the original statement and add a clarifying comment: if I have made a mistake, others might do the same.
Yes, this page is done in chronological order. I hate reverse chronological order. The latest entry is here.
Discussions on Australian lists suggest that I won't find a better alternative to computer video than MythTV or mplayer, so in the afternoon spent more time trying to get things to work. It's still like pulling teeth: spent all afternoon trying to work out how to get my tuner card set up. I've done this before with FreeBSD, but with Linux and even help from people on the list, it still took all afternoon. People on the lists suggested using xawtv to test the tuner; I've already commented on how bad that program's documentation is. It now comes with a new program called scantv. It prompts for standard and country and then does a scan of all channels. The problem is, it looks for a device called /dev/vbi, which doesn't exist. In FreeBSD, I can find out about devices from the man pages: for example, for a /dev/bktr I'll find a man page bktr(4). Linux, at least as I have installed it, has nothing corresponding. Google didn't help much either; the suggestion of creating a device with mknod based on major device numbers of several years ago didn't seem to be the right way to go. Finally found that there's a device called /dev/vbi0, so specified that instead, and it seemed to work. I still have no idea what /dev/vbi0 is. The name shows up in files like drivers/media/video/bttv-driver.c in the kernel, but it has almost no comments, certainly nothing to explain what vbi means.
The output of scantv was not surprising: I had no antenna cable connected, so at least for that reason it didn't find anything. I need to move it to where there's an antenna, but by the time I had got that far, I had no more time. Next week.
One useful source of information that I shouldn't discount is this diary; people read it and reply to me. Today on IRC, Olli Lehmann told me about the meaning of /dev/vbi. pointing to an article on zapping VBI. It explained the meaning of the name: “Vertical Blanking Interval”. Makes sense (the vertical blanking interval is often used to carry additional information). But what a way to find out.
More work on trying to get my TV tuners to work on ceeveear, the Ubuntu machine. It's still not easy. One of the strangest things I've seen has been that NFS mounting file systems from FreeBSD to Ubuntu takes 104 seconds per file system. Once mounted, access to the file system works fine. I suspected an incompatibility between the implementations, so spent some time running ethereal to try to work out what was going wrong, but that just showed a normal NFS mount operation followed by 104 seconds of nothing. The local (Ubuntu) /var/log/messages was more informative, but I no longer have the messages (see the end of today's entry); something about “unable to connect to localhost”. I suspect that it's a problem with the initial installation, but it happens on both machines (eucla, the laptop, and ceeveear, the TV machine).
The tuner was easier. I connected it to an antenna and answered the questions, and was gratified by:
=== root@ceeveear (/dev/pts/0) color="red">/home/grog color="blue">22 -> scantv -C /dev/vbi0
That all looks correct: we do indeed have channels 7, 9 and 10. But according to the man page,
So where's xawtv/fbtv? Not in the cwd, nor in the home directory. Since it didn't mention the pathname, decided that it might be somewhere else, and tried running xawtv anyway. Nothing, but the man page said that the configuration file was called .xxawtv, and not xawtv/fbtv. That wasn't there either. Ran scantv again and confirmed that it didn't write any output file. Then I read more of the man page:
It turns out that the text in the example above is in fact the configuration information:
Not that you can do anything with it like that; both the defaults and the man page are less than suitable.
Finally created a configuration file, and it selected channel 61 (conveniently, not occupied, so just noise on the screen), and discovered that I could not change channel: neither mouse nor keyboard worked. This seemed to be related to the messages it printed on startup:
On another attempt, it found a different channel, one that was in use, and showed that it was, indeed, receiving TV. But it still didn't accept any mouse or keyboard input. So, possibly a forgotten dependency? That seems common in Linux—see the installation of mplayer. Looking for related packages, I found:
=== root@ceeveear (/dev/pts/0) color="red">/home/grog color="blue">26 -> apt-cache search xawtv
So, which were installed and which not? What I wanted was the Linux equivalent of FreeBSD's pkg_info. Looking through the documentation for apt-get and apt-cache, that didn't seem right. Neither did the documentation for dpkg. So I went to the Ubuntu support site, which pointed me to a quick tour and a start guide, neither of which I wanted: I wanted a user guide. Looking on Google brought me 102,000 hits for "Ubuntu User Guide". The first one looked good, but despite calling itself an “unofficial user guide” (where's the official one?), it's a forum. Most of the other hits looked less than useful, so I limited my search to PDF documents. One single hit, not from the official Ubuntu site, and only 31 pages long, including table of contents, introductions and disclaimers. It had some stuff on software installation, but still nothing on how to find out which packages were installed.
OK, this system is laid out to work under GNOME, and there's a graphical program called synaptic. That produced its own error messages:
It also produced a ridiculously badly proportioned display, showing only the first third of the package category names, and that on a full screen 1600x1200 display. Fixed that, selected multimedia and looked for xawtv. Nothing. Also nothing in multimedia-multiverse and multimedia-universe. Only when I did a search for xawtv did I get a display that matched the list from apt-cache above, only alphabetically sorted. It showed that everything I could reasonably expect to install was already installed. So where is the problem? An hour of searching useless and non-existent documentation, and no closer to the solution.
At about this point, and not for the first time, I gave up. I decided that the hardware was working, but the software was not. I've had at least these problems with Ubuntu, and they shouldn't happen in a production release:
Not all of these have to be Ubuntu's fault, of course, and I'm reserving judgement on one other issue, which may be a configuration problem: when I open an X terminal on one of the FreeBSD machines that comprise my office desktop, the shell doesn't handle the Meta- characters correctly. For example, ~-F should move to the end of the following word. It does on FreeBSD, and it also does so on the default Ubuntu display; but with an Ubuntu X-term on one of my FreeBSD X servers, it capitalizes the word. Other keys are even less use. But that may be because of a discrepancy in character sets (UTF-8 on Linux, ISO-8859-1 (8 bit) on FreeBSD).
Anyway, I had just downloaded Fedora Core 5 from the Internode mirror server, so decided to try that on the spare partition (I always put two root partitions of 4 or 5 GB on my machines, and I only put system-related stuff on them, so that I can move from one release to another relatively easily. eucla even has three, two FreeBSD and a Linux, all of which share the same /home file system). Installed Fedora on ceeveear and was gratified by a much cleaner install. No problems with the meta-characters, NFS mounts Just Work, I now have a /dev/vbi, and I even found a MythTV HOWTO that seems to make sense, so followed that. Hopefully I won't run into Yet Another dead end.
Another interesting thing about GNOME is that I was able to get the top menu bar to display on wantadilla, my main machine, which runs fvwm2 as its standard window manager; this could be a good compromise between the GNOME menus and the flexibility of fvwm2. Today, though, it confused things: for some reason, it slowed down the repeat rate of the keyboard, and also disabled kbdcontrol, which should set the repeat rate. It also changed the input line editing on my local (wantadilla) firefox from Emacs-like to something Microsoft-like. But I know about that, and hopefully I'll be able to find a solution that is more convenient than anything I have so far.
Up early this morning with the intention to finally make progress with the Computer Video Recorder. I did make quite a bit of progress, but mainly towards getting functional Linux installations on ceeveear and eucla:
Following http://wilsonet.com/mythtv/fcmyth.php: For the SiS drivers for ceeveear, execute the following as root:
The directory http://www.winischhofer.net/sis/ contains a very large number of similarly named files, which firefox conveniently truncates. sis_drv.o_xorg_gcc3_current.tar.gz appears to be the latest version (i.e. not a static content).
At this point, things went wrong. /usr/X11R6/ has only one directory in it, bin. Fortunately, Fedora seems to do the right thing by locate (or maybe I was just lucky that it gets done Saturday night; I can't work out the format of the periodic cron scripts). It shows:
=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources/sis_drv color="blue">35 -> locate sis_drv
OK, rewrite that script:
Interestingly, though, that's still not the same name:
=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources/sis_drv color="blue">41 -> l /usr/lib/xorg/modules/drivers/sis_drv*
=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources/sis_drv color="blue">42 -> file /usr/lib/xorg/modules/drivers/sis_drv*
So which do we take? Took a look at Thomas Winischhofer's SiS chipset overview, which was instructive. ceeveear has:
On further investigation, http://www.winischhofer.net/sis/sis_drv.o_xorg_6.9.0_gcc4_091205-1.tar.gz contains the file /usr/lib/xorg/modules/drivers/sis_drv.so, and it's the correct version of Xorg, so I'll go for that.
While doing that, decided to set up the environment a bit better. One thing I definitely need is a window manager that I'm happy with, and that's pretty much fvwm2. Found the RPM and downloaded it, and was then forcibly reminded of why I didn't like Fedora:
=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources color="blue">102 -> rpm -i fvwm-2.5.13-1.i386.rpm
Spent a bit of time following up these references before it dawned on my that I was being silly. Some of these were missing simply because they were out of date. What I should have been doing was building from source—after all, when it comes to porting, I wrote the book. That worked relatively simply. Getting it to actually work was more complicated; I need to read more about GNOME.
At this point, the instructions became quite complicated, and I first need to read both them and the MythTV documentation to understand what needs to be done. The good news appears to be that there's a generic IR remote control application, which will make things much easier. But that's for another day. Tried installing xawtv, but again ended up with a maze of twisty little unfulfilled dependencies, all different. An attempt to build xawtv from source also failed, since I first needed to install the Xorg sources. Put that in the “too hard” basket.
Input from James Andrewartha on the topic of my Ubuntu woes. More in my diary, but this relates to CVRs:
Due to the way Linux development works, a lot of device information is only located in Documentation/ in the kernel source. DocBook/videobook.tmpl contains the following:
<entry>VFL_TYPE_VBI</entry><entry>/dev/vbi{n}</entry><entry>
The VBI devices capture the hidden lines on a television picture
that carry further information like closed caption data, teletext
(primarily in Europe) and now Intercast and the ATVEC internet
television encodings.</entry>
Back to looking at CVR software. Last Sunday I had got to the stage where I couldn't install xawtv on Fedora Core 5: the binary package had a missing dependency on libXm.so.3, and the source didn't build with some missing header files.
Investigated both. For libXm.so.3, the obvious choice was to find a yum repository with the library. It seems, though, that there are no lists of yum repositories—some of the web pages I found even seemed proud of the fact. Found a watch4tv page, which doesn't say so but appears to be a list of instructions to install a TV application on an operating system that uses yum. That was helpful, but didn't get me past the basic problem:
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles color="blue">48 -> rpm --import http://dag.wieers.com/packages/RPM-GPG-KEY.dag.txt
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles color="blue">49 -> rpm --import http://ftp.osuosl.org/pub/centos/RPM-GPG-KEY-CentOS-4
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles color="blue">50 -> yum install xawtv
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles 51 ->
Confirmed that I had a /usr/lib/libXm.so.4, but symlinking that to /usr/lib/libXm.so.3 didn't get me past the problem. Went back to the issue of building from source; further investigation showed that the missing header file belonged to the X font server, and that xawtv, though it uses autoconf, didn't seem to notice that the X libraries were not installed. Installing that made it build and brought instant gratification:
That's the TV image in centre left: it's tiny by itself:
It is indeed smaller than it should be, only 394x314. That's something to investigate, but the real issue was to get it to run on ceeveear. That didn't work as well:
=== grog@ceeveear (/dev/pts/11) ~ color="blue">3 -> DISPLAY=:0 xawtv
That looks like a problem with the X configuration; maybe the HOWTOs will help there. Also the program scantv that I used last week under Ubuntu is no longer built by default; the sources are there, but when I try to compile them, I get lots of errors. More stuff to investigate. This whole exercise reminds me of when I was writing Porting UNIX Software well over ten years ago. Why do these problems still exist?
Yvonne back with more mushrooms today, along with the information that she had met some Polish women in the forest who were gathering large quantities. They offered to tell her the name in Polish, but she didn't bother. I sent a message out to an internal MySQL list and got answers from Mårten Mickos and Boel Larsen, who both apparently eat them and tell me, as did Alexander Keremidarski, that they're not boletus pinophilus, but suillus luteus, called smörsopp in Swedish and maslovka in Bulgarian (also Butterpilz in German), all of which mean the same thing. Certainly the photo looks convincing enough. They need peeling, apparently, and there are some indications of people who get stomach upsets from them, possibly only if the skin is not removed.
Today continued my attempts to install MythTV. First continued with xawtv, though, and looked for scantv, which I have used before. It wasn't there. There's no target for scantv in the top-level directory, but there's a file console/scantv.c that won't compile in that directory:
=== root@ceeveear (/dev/pts/1) color="red">/usr/src/ports/xawtv-3.95/console color="blue">14 -> make scantv
Presumably config.h was in the top-level directory, so tried it there; it got a bit further:
=== root@ceeveear (/dev/pts/1) color="red">/usr/src/ports/xawtv-3.95 color="blue">16 -> make console/scantv
With a bit of investigation (including far too much guesswork), established that this related to libzvbi, and that the corresponding package was called zvbi. Installed that and zvbi-devel, but scantv still didn't compile. It looked as if the header files weren't being found; but which header files? Spent some time trying to get yum to tell me which files were associated with the packages that were installed (something like FreeBSD's pkg_info -f), but that doesn't seem to exist. It was also strange that scantv.c didn't complain about missing header files. Then it dawned on me: configure was run without the libraries being present, so it decided not to build the programs. Re-ran configure and all got built.
Next was the problem with saving the config file. I had already discovered that scantv writes its output to stdout; but what's the real config file called? xatwv(1) refers to the file several times, but refuses to divulge its name. That's in xawtvrc(1): it's called ~/.xawtv, a funny name considering the name of its man page.
Finally moved on to lirc, the Linux Infrared Remote Control software. That was a new level of incomplete, inaccurate, incorrect and confusing documentation. According to one page, I should try irrecord to find out what the control said. That resulted in:
=== root@ceeveear (/dev/pts/0) ~ color="blue">219 -> irrecord foo
Well, lircd wasn't running, so I tried strace:
=== root@ceeveear (/dev/pts/0) ~ color="blue">220 -> strace irrecord foo 2>&1 | less
What kind of programming is that? It doesn't even check the return value from open! Of course, I don't have the source, so I can't do the obvious two-line fix, and getting it is liable to cause other problems.
In the middle of this, ran into another problem: the dreaded NFS hangs, this time between ceeveear and wantadilla. I had to reboot ceeveear to get out of it, and then X wouldn't start: the font server kept crashing. strace didn't show anything of interest, so modified the X configuration to load the fonts directly (why are they stored in /usr/share/X11/fonts and not in the usual /usr/X11R6/lib/X11/fonts? Is this a change that came with X11R7?).
After starting X again, discovered that I also no longer had a window manager. That was GNOME anyway, something I don't need, so changed the init level to 3 and started fvwm2 manually, the way I'm used to do it.
Then continued for a while, discovering that my tuner card (MSI “TV@nywhere Master”—where do they get these names from?) probably required a kernel module (I was left to guess which one), and that lircd require a “driver” (again, though there's an entry for my remote control, there's no information about which driver to use). Finally went through with a brute force search and found that a number of them worked, including at least bte and dsp. Of course, they only understood some of the keys; for tomorrow I need to investigate how to decipher the keys.
For the fun of it, tried to run xawtv again. The results were startling: the X server crashed with a SIGSEGV and a backtrace:
I wonder if the NFS crash somehow ended up corrupting some of the font files. I've never seen that on FreeBSD, but I've heard of it on Linux, and recently I've been experiencing lots of things I have never seen before. The overall fragility of this system is horrifying, a far cry from the claims of absolute stability that the die-hard “Open Source” advocates make.
Tried more stuff with lirc, but it didn't behave as described in the “manual” (not the man pages; that would be too simple. Instead it's on the web). After several readings of this document, I still don't know what to do.
Finally downloaded the source, which also required the kernel sources. ./configure was a surprise: I got a menu asking me which tuner card to compile for (and of course my TV@nywhere wasn't on the list). Somehow it's possible to generate the programs with all “drivers”, but it's not made clear how. It's also full of references to other documentation, which in turn leads on to yet other documents, none of which seem relevant. After following the instructions, I ended up with the Makefile configured only for a serial connection. After a lot of investigation, I found:
1. Which kernel module do I have to load for my hardware?The documentation contains a detailed list how to setup the various devices supported by LIRC.
Somehow I then found my way to the installation instructions at http://www.lirc.org/html/install.html#compiling, which state:
See ./configure --help for a detailed description of the possible parameters. You will have to at least choose a driver with the --with-driver=X option.That worked: ./configure finished with the message
Of course, it didn't compile:
Round about here something strange occurred to me: the system was responding to the IRC control even when there was no lirc program running! Further investigation showed that there was a kernel module ir_common, and trawling the kernel sources showed that the driver does indeed understand the remote control interface; found some useful information at LinuxTV.org, which enabled me to—finally!—configure the thing.
The next step was easy: install MySQL. Including setting up the MythTV databases, it took about a minute. The next step, mythtvsetup, looked complicated enough that I decided that it would be better to put it off until tomorrow.
On with the pain that is MythTV. Started writing a HOWTO so that I'll find my way better next time. That might be sooner than I expect: I got as far as the step Run mythtvsetup. It's not a very clever program:
Setup finally ground to a halt with the message: Could not open '/dev/viso0' to probe its inputs. Stopped it to run strace, and saw, amongst other irrelevant messages:
=== mythtv@ceeveear (/dev/pts/2) ~ color="blue">3 -> mythtvsetup
I wonder what all that means. It's the first I've seen of a ~/.mythtv/lircrc.
Running strace showed what you might expect:
I feel like screaming! Why can't people report errors any more? But even after I fixed the permissions on all likely devices, mythtvsetup could not access the card. Tried to run xawtv again. It still crashes the local X server, though (modulo network speed) it works fine on wantadilla. Discovered that even on wantadilla, xawtv didn't work for user mythtv; it just displayed noise. Copying ~grog/.xawtv to ~mythtv helped, but only after an explicit channel change. This software is so buggy!
Decided to test my installation instructions and possibly get rid of some crashes by installing Fedora Core 5 again on a new (well, very old, but overwriteable) disk. In the process, various niggles became apparent:
While waiting for that, also downloaded a CD with the of KnoppMyth, which doesn't make it easy. Most of these documents look really confused, and neither the home page nor the FAQ delivered an obvious answer to the obvious question: where can I get it from? The answer is in http://mysettopbox.tv/knoppmyth.html, starting with the line “R5B7 is available from:”; that's the name of the latest version, spelt differently from the previous time it was mentioned.
KnoppMyth states at several occasions on the way that it's experimental beta software; most versions of Knoppix (from which it is derived) do something similar. In each case, they're wrong; Knoppix is surprisingly stable and, bar the horrible eye candy, not at all a bad installation. KnoppMyth is barely alpha quality. The installation scripts forgot the passwords I entered and chopped the first characters off each IP address. They then put me back in the same position I was earlier in the day; the only advantage was that failure came more quickly.
Very frustrated, gave it up for the day. This is not merely a question of compatibility, as some people say; it's a question of really badly written software. Not checking system call return values is a sure course for disaster.
On trying to debug ceeveear today. There's definitely something wrong with the installation for Fedora Core 5: I went to some trouble to specify that the X11 sources should be installed, but they weren't. Emacs was also missing, but it's possible that the packager is a vi bigot and left Emacs out of the default editor installation.
Finally got it installed, along with all the bits and pieces needed to build xawtv. It didn't crash the X server, but it did the same thing as KnoppMyth—blank screen and lots of ioctl error messages:
It did this at each attempt to change channel; paradoxically, though there was no image in the main window, it displayed an image in the channel window, suggesting that there's some issue with the phone.
That left the question why the X server crashed before. Installed the new, corrected xorg.conf file and the new driver and—it crashed! So it wasn't something I did with the installation on the “old” version of ceeveear.
In the meantime I had downloaded a CD of FreeBSD 6.1-RC1, so installed that on a spare partition, in the process finding out again how to boot FreeBSD from grub:
The important thing to remember here is that “hd0, 0, a” corresponds to the FreeBSD name ad0s1a; the BIOS partition numbers are off by 1. This isn't what Fedora puts in by default, and it had caused me problems booting eucla, my laptop.
It ran (of course), but didn't detect the tuner, as I expected, so I still need to continue with Linux.
Rebooted the “old” ceeveear and played around with xorg.config. The old one didn't work at all, of course, because it relied on xfs (font server; what a confusing name), which kept crashing. After much trial and error, discovered that the crashes occurred only when Type 1 fonts were used, this entry:
Without that, it ran fine and displayed the same ioctl problems as KnoppMyth and the “new” Fedora installation. So it looks like it's a display card problem. Spent some time investigating that, without coming to a conclusion.
Funnily, there seem to be multiple issues with xawtv. It continues to run across the network on echunga and wantadilla, my main machines in the office; but it won't run on teevee, the machine that (currently) drives the projector, also a FreeBSD machine. Another thing that I don't understand.
Spent more time searching for the ioctl errors that had been annoying me yesterday, and got quite a few results on Google. Many reported the bug and got no solution. Some reported fixes that were really workarounds (use a different tuner card, for example). Michael Krufky reported that it was a kernel bug in Linux 2.6.15-rc2 and before; unfortunately, I'm still getting it in kernel 2.6.16. Finally, Hermann Pitton came up with a plausible explanation: the driver doesn't support overlay with PCI/PCI transfers to the graphics card. That would explain why it only happened when displaying on the same machine. But why can't there be a better error message?
Hermann recommended to use xawtv grabdisplay and referred to a file called README.cx88 without saying where to find it. It wasn't on ceeveear, and it's not in the xawtv distribution; nor is there more than a passing reference to it in the documentation. More googling suggested that it's in the Linux kernel sources, and indeed I found it there on echunga. But there's no mention of grabdisplay in that file, just the comment “For now only capture, overlay support isn't completed yet.”.
Found a reference to a program called tvtime, and installed that. Oh wonder! It just worked. Well, I had to read through the config file that was supplied, and it got a bit confused about the channel numbers, but it worked, modulo sound, which doesn't seem to be an tvtime problem: it doesn't work at all yet. About the only thing wrong with it is that it only displays TV. If only more software were of this quality.
Spent some time investigating the sound issue. According to man -k, there isn't much choice in mixer software:
=== root@ceeveear (/dev/pts/3) ~ color="blue">46 -> man -k mixer
I suppose there's more, but how do you find it? They could at least have extended man to show what other documentation is available.
Running alsamixer brought a message that I've seen before:
=== grog@ceeveear (/dev/pts/6) ~ color="blue">6 -> alsamixer
What device? Are people deliberately trying to make it more difficult? strace revealed:
After fixing the permissions, it seemed to work, but it wasn't very easy to understand. Tried the HELP menu, which told me:
That doesn't work, so I of the help menu. tried ESC. That got me out quickly:
=== grog@ceeveear (/dev/pts/5) ~ color="blue">2 -> alsamixergui &
=== grog@ceeveear (/dev/pts/5) ~ color="blue">3 ->
I think I'm going to modify that message from bash to
By default, a SIGSEGV creates a core image; if it doesn't, it's because the program has explicitly decided not to. Considering that much of this software is of early alpha quality, this just makes it even more difficult to debug. Decided to read the documentation again before doing anything else, and returned to trying mythtvsetup again.
After 5 minutes of swearing at the completely broken interface (right arrow moves left, left arrow moves right, cursor not visible), gave up and looked for a simpler way, like writing a program to do it or encoding it in hex. I didn't find one, so here's a list of things to remember:
Round about this point, my cousin Sandy and her husband Dean showed up. They live just down the road, but we haven't seen them since December 2004. I suppose we should stay in better contact. At least it gave me a chance to get away from this multimedia frustration.
In parallel with the other work, started downloading the latest (DVD) version of Knoppix from the Internode mirror server; contrary to my normal use of command lines, I this time allowed firefox to do it. That always annoys me, because firefox is just completely stupid about path names; it started downloading to Desktop, which on this machine proved to equate to /var/tmp. There was enough space there, so I let it go. Some hours later I found it had stopped:
=== grog@echunga (/dev/ttypl) color="red">/var/tmp color="blue">57 -> l KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso
That wasn't the real size, of course; the download had stopped at the 2 GB limit. Grr. And because I was loading from a web server, I couldn't use REGET. Looked around on the web and found alternatives, and started that transfer, somewhat slower (50 kB/s instead of 150 kB/s).
Came into the office to discover that my Knoppix download was still going:
That's nearly 24 hours. Network transfer speeds may have increased, but so have download volumes. Did some investigation and discovered that the Internode mirror server also has an ftp interface, so finished off the transfer from there.
Today was the last day of a ten day period during which I had intended to do a number of things; just about all of it was spent fighting Linux multimedia, and I'm still far from finished. The current biggest problems are:
Another holiday today, so spent more time torturing myself with multimedia setup. A package that I haven't tried so far is VLC, so set to trying that out. Found a detailed (and somewhat opinionated) description by Brent Norris, which required a complete kernel upgrade. That didn't go well. Once again dl.atrpms.net was not accessible. It's a real pain that you can't tell yum to ignore it; it causes the entire update to fail, even if you tell it to tolerate errors. I had to move the file away (yes, I know you can also disable it, but it's easier and less error prone to move it). Even then, I got messages like this:
But Emacs was installed, for example, and so was libgif.so.4. While experimenting, removed Emacs:
=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog color="blue">64 -> yum remove emacs
=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog color="blue">65 -> yum install emacs
Then, later, it happens again:
There's obviously something very wrong here, but it's not clear what I can do about it. Later I got, after downloading 230 MB of packages:
No idea what that is. Checked things, but it wasn't obvious:
=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog color="blue">71 -> echo $?
=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog 73 -> df
So it wasn't a lack of space. The annoying thing is that yum appears to insist on downloading every package every time; presumably they're already on the system. With 230 MB at a time, it takes a while, and it could become expensive. Ended up installing some of the larger packages individually; I think in future I'll always do it that way.
That still didn't help too much, of course, since dl.atrpms.net was still not accessible. Tried some other things and with a lot of effort more or less got mplayer to display on the screen, something that tvtime had done out of the box.
=== grog@ceeveear (/dev/pts/1) ~ color="blue">3 -> mplayer -tv driver=v4l2:normid=2:width=960:height=720:chanlist=australia:\
channels=2-ABC,7-Seven,9-Nine,10-Ten,28-SBS:outfmt=yuy2:brightness=70:saturation=90:hue=50 -vo x11 tv://5
Unfortunately, getting the companion mencoder to make MPEG files out of that didn't work: mplayer claimed that the result was not a valid MPEG stream. What a crock!
Other events included installing the DVB-T tuner that I borrowed from Ben Kramer on Thursday, along with c't magazine's Debian “VDR” software. I think this stands for “Video Disk Recorder”, a term which doesn't make much sense to me: where's the video disk? I prefer to use the more descriptive term “CVR” (“Computer Video Recorder”). That hung up a surprising number of times; I wonder if the motherboard is getting even flakier. And why does Debian load the floppy driver three times in the course of installing on a system that doesn't have a floppy drive?
Spent the morning writing up some recipes, then continued trying to install c't magazine's “VDR” software. I think there must be a conspiracy amongst authors of multimedia software to avoid documentation; once again, and unexpectedly for c't, the documentation was almost completely missing. During the installation I had to choose a transmitter profile, and of course they were only available for Germany (I didn't even see Switzerland or Austria). The on-screen instructions (all that I ever found) said that they could be changed afterwards, so chose one at random. At the end of the installation, the terminal under Alt-F8 was supposed to have the application; but it just produced a text-mode message saying (roughly) “this screen reserved for DVR”.
Spent a lot of time investigating and discovered:
Configuration information is spread widely throughout the wiki, so this page is just first aid for finding where to start searching.It then points to the user handbook, which starts by describing how to use the software in everyday situations. I still haven't found anything that describes initial setup.
Also tried KnoppMyth again. This time the installation failed consistently with an error in an installation script. I'm not sure what I did differently, but given my previous experiences, I couldn't be bothered trying more than 4 times. Gave up in disgust. This is all far more complicated than it needs to be.
Put the new motherboard in ceeveear. It's not at all new; in fact, it's so old that it can't correctly identify the Athlon XP+ 2500 processor that I'm using, and it runs at half speed. I've also had to scrape up some SDRAM because it doesn't take DDR. But it runs, and the hangs I had before no longer occur. I still couldn't get the “VDR” software to do anything useful, though, and this brain-dead setup is such a pain that I gave up after installing X and discovering that the configuration routines couldn't recognize my nVidia 6200-based board and wanted all the horrible configuration rigmarole that you used to have to supply years ago.
Tried KnoppMyth, which installed this time (another difference from two weeks ago), but it forgot to install the boot loader. What a pain this stuff is! Installed FreeBSD 6.1-PRERELEASE and started building -STABLE: at the speed this processor is running, it took the rest of the day before hanging on NFS access to echunga. That never used to happen.
More work on ceeveear, the computer video recorder that I'm trying to build. Got a fractionally newer version of KnoppMyth, which refused to exit the partition menu during a manual install. Gave up on that, grabbed another (“empty”) disk and installed on that using the “automatic” install. That worked, and it even recognized my nVidia 6200-based card and installed a driver on it which set contrast so flat that I could barely recognize anything. Put in an nVidia 4000-based card and it worked. MythTV still has one of the strangest user interfaces I have ever seen, and I still wasn't able to get it to “open” the video card. xawtv found it (though at the time I didn't have an antenna connected), so it looks like my next job is to learn how to set up MythTV.
Daniel O'Connor along in the afternoon to pick up a tuner, some Weihenstephan W68 yeast and the old iiyama monitor that I retired on Thursday. In turn he brought an AMD Duron 800 processor, still mounted on a motherboard of dubious quality:
After that, finally got the motivation I needed to continue with the work on the video recorder project. The last time I tried this was only barely less frustrating than the previous attempts, though there were signs that it might be more promising. On IRC and the LinuxSA mailing list I had got some suggestions to try tzap, which was typical of the documentation standards for multimedia programs:
=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images 11 -> tzap
=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images 12 -> man tzap
=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images 13 ->
So I went googling for tzap and finally found a wiki page describing it; it seems that I couldn't use it until I had run dvbscan, which of course also had no documentation, and even the wiki page was pretty cursory. It did tell me that I needed a list of local transmitters, though.
Based on the information, extrapolated a file name au-Adelaide, went googling again, and to my surprise discovered indications that I should already have the file on the machine; and I did:
=== root@ceeveear (/dev/ttyp1) ~ color="blue">26 -> locate au-Adelaide | xargs ls -lrt
With a bit of playing round, got the tuner to work and copy MPEG data to disk. To get that far, I needed:
=== root@ceeveear (/dev/ttyp1) color="red">/myth/Images 58 -> dvbscan \
/usr/local/share/dvb/scan/dvb-t/au-Adelaide > channels.conf
It's not clear what the error messages at the end mean; I couldn't find anything corresponding in the input file, and dvbscan found everything I expected it to find.
Find the names of the channels from the channels.conf file. This file is a typical UNIX colon-delimited file, and the channel name is the first field:
=== root@ceeveear (/dev/ttyp1) color="red">/myth/Images color="blue">60 -> tzap -r ABC HDTV
=== root@ceeveear (/dev/ttyp1) color="red">/myth/Images color="blue">61 -> tzap -r "ABC HDTV"
Once again spaces in names rear their ugly head; it is possible, though, to write abc hdtv: it obviously does a case-insensitive search. The indication FE_HAS_LOCK at the end of the lines indicates that the station is “tuned”, and the -r option enables recording output via /dev/dvb/adaptor0/dvb0 (in this case; the values of 0 may vary).
From another xterm, copy the data to disk:
=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images color="blue">13 -> cat /dev/dvb/adapter0/dvr0 > abchdtv.mpeg
Today messed around trying to get MythTV working. As expected, it wasn't as simple as it should have been. Although they're mentioned on the web site, tzap and friends have nothing to do with MythTV, and the files they use are of no use in setting up MythTV. In addition, the documentation is very weak; it doesn't really tell you what you need to know to configure the tuners. From mythtv-setup, fortunately started from an xterm, I discovered that the cause of the hang was that it started a command on the xterm. I wonder where that would have gone if it had been started from the window manager.
All I managed to do was 4 (“Free to Air”), which gave me a file ~/.mythtv/Tuner-1.xmltv with the following contents:
That's clearly analogue. I couldn't find any way to get the digital signals. The same happened with “Free to Air Digital (High Definition)”. Gave up and sent a message to the local mailing lists asking for help.
Trying to configure mplayer to display the data I copied yesterday was no better. mplayer did completely different things depending on where it was started from and where it displayed. In all cases, I got the error message:
This, though, is just “ha ha, only joking”; it doesn't stop mplayer from working. What I got depended on where I ran the program and where I directed the output:
This is the configuration in which I've been watching video from DVD for about a year; normally I see:
The only difference I see here are the chosen output dimensions.
By comparison, the same program output the same file on wantadilla:
Only here, the aspect ratio was messed up, apparently by a factor of 2 to 1.
Finally, in the evening, I tried looking at the
This turned out to be because of the -alang en parameter that I also provided; that works for DVDs, but not for the input stream, but I didn't find that out until later. Wouldn't a descriptive error message be good?
As usual when writing up my diary entry for the previous day, I tried reproducing some of the problems I had, in this case including the inability to set the channels. To do so, I started mythtv-setup on wantadilla:0.0, the 2048x1536 display. This takes forever! So did key presses, with the result that I accidentally selected the wrong entry, didn't notice it, and found myself in the channel scan menu. Despite all the error messages from yesterday, it seems that it had found the information it needed, and it was able to perform a (surprisingly slow) channel scan and locate the channels:
Went looking for the ~/.mythtv/Tuner-1.xmltv file, but there was none. After a bit of searching, discovered that Chris Yeoh was right: the channel information is stored in a MySQL database, specifically the table channel in database mythconverg:
=== root@ceeveear (/dev/ttyp6) color="red">/myth/Images color="blue">31 -> mysql -D mythconverg
Getting closer to work, made a dump of the table with mysqldump; presumably it's the same for all installations in Adelaide, though notably the frequency information is missing.
In the evening, confirmed that the sound problems yesterday were caused by the audio language settings of mplayer, so finally was able to play the MPEG stream with sound. Even then, though, the problems aren't over. Trying to play the recorded HDTV, the recorded audio failed again:
So what's going on there? It really looks as if there is no audio in the feed. I'm not going to worry about it too much until I get MythTV working and it shows up there too.
As if to add to the general instability I've been seeing lately, tvremote, the laptop I use to control teevee, died on me. Investigation showed a loose mains plug adaptor. sigh I think that machine had been up for 6 months.
Today finally found both the time and the courage to look at the question of recording video from my DVB-T card. As I had mentioned last month, tzap offers me the opportunity to record without further pain installing undocumented software. But it would be nice to have a wrapper program to control tuning, start and stop. So I wrote a little program called record. Most of the code is, not surprisingly, in the time routines—why is UNIX timekeeping such a mess? Ended up writing about 800 lines, very little of which had to do with actually starting and stopping the tzap process. And it worked. Well, almost. Stopping the process on occasion caused the hardware to get confused:
This wasn't the first time; the log files showed that it had happened before in March. I didn't find a way round that beyond rebooting; hopefully it won't happen if I don't kill the tzap process.
In addition, the recording stopped after 2 GB:
Why? There's nothing in this file system that has a 2 GB sensitivity. Tried looking in the man pages:
=== root@ceeveear (/dev/pts/0) color="red">/var/log 46 -> man 2 open
=== root@ceeveear (/dev/pts/0) color="red">/var/log 47 ->
I could scream! Why do Debian consider documentation optional? On eucla, my Fedora machine, I found:
But that's only for LFS, and a test confirmed that the flag wasn't even known.
Still, despite all, the day's work was a success. It's a sad recognition that it's easier to write your own program than install somebody else's. Documentation standards seem to have reached an all-time low.
Chasing up the failure of record to write beyond 2 GB was interesting. First, I found the Debian development man pages and installed them. man 2 open still gave me nothing; the man pages was installed as /usr/share/man/man3/open.3posix.gz, which leaves me wondering if it's even the right man page. It says nothing about O_LARGEFILE (not surprising; it has nothing to do with POSIX.1). Looking for the flags in the required header files sys/stat.h and fcntl.h brought nothing: the header files are part of the incredible mess that is the Linux system header files. The actual definitions are hidden in a maze of #ifdefs that ultimately include /usr/include/asm/stat.h, which defines three incompatible structs, __old_kernel_stat, stat and stat64. struct stat has offsets that are unsigned long (i.e. 32 bits on i386), so that can't work.
Spent some time talking with Linux people, and finally came up with the fact that I needed to define _GNU_SOURCE to get the correct definitions, and then add O_LARGEFILE after all, and replace calls to fstat with calls to fstat64, and define the receiving structure as struct stat64. And then it finally worked. But what a mess! I don't think even the most convoluted System V headers were this bad.
More importantly, though, the man pages say nothing about this mess, neither a definition of the structures, nor a mention of the _GNU_SOURCE requirement, nor the restrictions on file size. By contrast FreeBSD is a model of cleanliness. About the only excuse I can find for Linux (and a feeble one at that) is that this is the price of compatibility, exacerbated in Linux' case by the fact that the kernel and userland come from different places.
Mail from James Andrewartha today, pointing out the mistake that I had made yesterday: I had installed the wrong Debian man pages (manpages-posix-dev instead of manpages-dev). Under those circumstances, it's not surprising that they're in section 3 and don't refer to Linux specifics. But it's easy to make this mistake: apt-cache search manpages gives me 65 different packages, and it's easy to miss this:
The real issue is that man pages shouldn't be something you have to go out and get. If you install development software, the man pages should be there too. That was one of the great new ideas with UNIX over 30 years ago. In those days, the man pages took up a significant amount of the available disk space, but people obviously thought it was worth it; nowadays it's just noise.
The manpages-dev pages appear identical to the man pages on my Fedora system, as you'd hope. As I've noted, they mention O_LARGEFILE, but nothing else. James also found a reference to LFS on Andreas Jaeger's web pages. It seems that LFS stands for Large File Support, not Log-structured File System. Somehow it's all not very satisfying: first, there's no need for this stuff, and secondly if it's needed, it should be documented in the system, not somebody's web site.
Spent some time playing around with record, the DVB recording program I wrote on Sunday, making it more user-friendly (for command line users, of course). One of the interesting things is that Linux doesn't have a setproctitle() function, which changes the text displayed by ps(1). In the end, found a way to fake one by overwriting the argv and envp parameters at the top of the stack:
The results are visible in the ps output:
=== root@ceeveear (/dev/pts/1) color="red">/myth/Images color="blue">172 -> ps aww | grep record
More investigation of the sound problems with HDTV, with some interesting non-results. I need to learn more about MPEG-2, but it's clear that multiple data streams are multiplexed in packets, and that each packet header contains a package ID or (confusingly) pid defining the stream to which it belongs. There seems to be no particular standardization of the pids for audio and video. mplayer thus investigates the beginning of the stream to determine the pids involved in the stream; somehow it also manages to determine their nature, by a means I haven't investigated yet. With the -v option, it shows what it chooses. For Channel 10 TV (normal definition), I get:
By contrast, 10 HD gives:
So it didn't find the audio pid. But the video pid is different too; it's different on other channels as well:
| Channel | Video pid | Audio pid |
| 7 | 1281 | 1282 |
| 9 | 512 | 650 |
| ABC | 512 | 650 |
| ABC2 | 2307 | 2308 |
| SBS | 161 | 81 |
| SBS 2 | 162 | 83 |
| SBS HD | 102 | 103 |
Clearly it's difficult to guess the pid, especially since it can get quite big. But ~tzap/channels.conf contains lots of information. Could it include this stuff? Looking at it (and removing a lot of stuff in the middle), I found:
Clearly the third last and second last columns are the video and audio pids. But what's the last column? It looks like a pid too, especially since exactly those channels which I can't get audio for don't have an audio pid, and SBS radio (the last line) has no video (not surprisingly) and the last two columns both set; in fact, all channels, including the ones I don't have set here, have the last column set.
So what is this? Some kind of digital encoding, MP3 maybe? The abbreviation MPA in the successful case may be a clue. The obvious thing to do is to read the tzap documentation. But, as I have complained in the past, that doesn't exist. OK, find the source. That's also more easily said than done, and by the evening I hadn't foudn anything up-to-date. apt-get said:
=== root@ceeveear (/dev/pts/1) ~ color="blue">265 -> apt-get source dvbutils
That will have to wait for another day.
In the afternoon, more investigation of the “no sound” problem on HDTV. Spent quite some time determining that yes, it seems that I did have the latest version of dvb-utils. Documentation is horrendous: even the name of the package is not clear. Debian calls it dvb-utils, but elsewhere it's called dvb-apps or dvbtools, and the package itself is called linuxtv-dvbtools. After a lot of searching, discovered the cause of my problems: the audio modulation for HDTV in Australia is different, but the information I need is not present at all in the channels.conf file: the last field is the service ID, whatever that may mean. Instead, HDTV uses (only) AC3 audio encoding (the other programmes also apparently do so), and dvbscan doesn't look for that when run in the “build channels.conf” mode. Instead, you need to tune to the stream with tzap and then run it in a different mode:
=== root@ceeveear (/dev/pts/0) color="red">/myth/Images color="blue">262 -> tzap -r 'ABC HDTV' -S &
=== root@ceeveear (/dev/pts/0) color="red">/myth/Images color="blue">263 -> dvbscan -c
Conveniently, this output is also in hex; presumably somebody else wrote it. Putting the AC3 PID into channels.conf finally gives audio. But what a pain to find that out!
More work on the stuff I learnt yesterday, mainly updating the LinuxTV wiki with pages for dvbscan and tzap. In the process discovered that we're not out of the woods yet; sometimes mplayer still doesn't find the audio in HDTV streams.
Playing HDTV also brought me up against another issue: it looks like my machine isn't fast enough, and I get lots of dropouts:
Tried running it on eucla under Fedora Core 5, where it didn't have any problems with dropouts—but exactly every ten seconds it hung for about 5 seconds, and then continued as if nothing had happened. I suspect that's some artefact of GNOME, so rebooted the machine into FreeBSD to try it there—but of course mplayer wasn't installed. Spent some time trying to do that, but didn't finish.
Finally got mplayer installed on the FreeBSD incarnation of eucla and confirmed that it didn't have any difficulties with the HDTV MPEG stream. So the simple answer seems to be: throw more CPU at the problem. eucla has a 2.0 GHz Pentium M; teevee has a (1.466 GHz) Athlon XP 1700+.
(Not directly related, but potentially of interest):
Playing around with the cxm driver for Hauppauge PVR-350: we found that if you pipe it directly into mplayer, it suffered severe buffer overflows, giving really bad images. Modifying the number of buffers in the ring significantly improved that; we'll have to look at what's really needed.
Also looked at the mplayer port, about which I have said less than friendly things in the past. I have only just discovered that the version in the Ports Collection is version 0.99.8.2, which is so old that I can't even find a date for it; but 1.0pre7 came out well over a year ago, and 1.0pre8 came out recently. Did some attempts to build that, with the usual problems of port dependencies getting in my way. Put it on quartet.lemis.com, a test machine with (almost) no ports on it, and left it go (it's a quad CPU 200MHz Pentium Pro).
Writing up my diary entry for yesterday gave cause to investigate the versions of mplayer I was running. Looking at the build log file, I found:
So it is the latest version, only the reported version number is wrong!
Following up on my HDTV sound problems of a few days ago, discovered that a programme I had recorded from 9 HDTV still claimed to not have any sound. I'd seen intermittent problems in this area, so kept the program for investigation. As it turned out, it worked fine on wantadilla, but not on teevee, which is running 5.3-RELEASE. It also has problems with a bad sector on the main disk, so it's a clear candidate for an upgrade. I don't have a spare machine, and I don't want to take teevee down for an extended period, so chose ceeveear instead—the only thing that it's doing is recording the occasional programme, and I can always reboot it for that. Chose a new disk and installed 6.1-RELEASE, then upgraded to 6.1-STABLE. In the process it came home to me just how little I have achieved in the last two year in my attempts to create a good system upgrade procedure.
Planned a quiet day today, but somehow it didn't quite happen that way. Instead, continued with the rebuild of teevee, which proceeded slowly but steadily and took up most of the day. In the end had swapped the hardware of ceeveear and teevee. As a result, I can now watch HDTV with audio and only use about 50% of the CPU. In combination A number of things have probably contributed to this:
Yvonne into town this afternoon to pick up some pieces for a new, real CVR, based on a “bare bones” system from MSI. Amusingly, this relatively low-end machine is my first own AMD64 machine (I had a dual Opteron machine last year, but it belonged to Rocksoft). The current box looks interesting; it's almost the size to fit into a home entertainment tower, though I fear it might be a little deep. Hopefully it won't be too noisy.
Spent most of the day today installing the new CVR machine. Its internal layout is quite interesting: there's a removable cage at the front for disk and DVD drive (and, puzzlingly, for a floppy disk, made in such a way that you can't use it for a second disk). The remainder looks quite well laid out too:
Putting the machine together wasn't without its surprises. First, I ended up with the wrong memory (DDR-2 instead of DDR, so I had to take some memory out of teevee), and secondly I found two fans, one that came with the CPU and one that came with the system:
The (larger) one on the left is just sitting on the PCI slots for comparison: that's the one that came with the CPU, but it doesn't fit properly on the CPU (too loose). I wonder how many people have lost their CPU as a result, and why MSI made such an incompatible board.
Installing FreeBSD on the machine brought a couple of surprises, too: read errors from the installation DVD, and later a panic with corrupted data. Guessed at memory incompatibilities, but the SIMM I put in was a PC 400 SIMM, and that's what the BIOS was set for. Turned the memory speed down to 166 MHz and had no further stability problems.
Setting up X was the next issue: the board has its own display chip, but x.org doesn't recognize it. It managed to set it up with a really unusual resolution, something like 1800x1350, and mplayer wasn't able to render correctly on it. Put in an SiS 315 board, set to 1280x720, and it worked.
But, not for the first time, it seems that 64 bit machines are slow! This is an Athlon 64 2800+, presumably intended to be faster than the Athlon (32) 2500+ in teevee; but tests with a specific HDTV file showed much more CPU usage. Of course, there are a number of things to compare: display card, display drivers, memory timing and 64 bit mode. I'll do them in sequence.
I had intended to go riding today, but it was cool, and somehow I managed to find excuses not to do so. Instead carried on looking at the new CVR machine, and did numerous tests with mplayer, making interesting discoveries on the way:
In summary, nothing I did significantly improved the situation. The next step was to install a 64 bit kernel; but that failed miserably. The instability I had noticed before became impossibly bad with the 64 bit kernel. Both FreeBSD and Ubuntu didn't make it through the installation process. Spent a lot of time playing with BIOS settings (the “restore default settings” and “set optimized defaults” both seemed to do nothing), and finally removed all boards, used the new disk I had bought for the machine (previous attempts were done with disks about 3 to 5 years old, but still in good condition) and tried again—and it worked! So: is this a problem with the BIOS settings, some of which I might have reset after all, with the boards, or with the disk? Who knows? Still more work to do.
Brew day today, but also found some time to work on the CVR machine. Spent a lot of time working on the system upgrade scripts, but also installed X—strangely, it created a configuration file very different from the one it created on the same hardware in 32 bit mode. Did some preliminary tests with mplayer suggested that it was a lot faster than in 32 bit mode.
But before I could start some real tests, it crashed. Not once, but repeatedly; it took several attempts before it would stay up long enough to finish the boot and fsck. And the only way I could get it to stay up longer than about 5 minutes was to turn all the timings down to their minimum values, which also slowed the machine down unacceptably. Finally, frustrated, gave up. What a waste of time!
What a waste of a day! Spent most of it double and triple checking what's wrong with the new machine, without any firm result except to confirm that it's not working correctly. That's a particular nuisance because it's holding me up doing other things, notably video performance testing.
Into town with my defective computer today and spent quite some time at the warranty/service department, where I was first told that the processor that I had been given was a 32 bit Athlon—this after I had said that it had run for some time in 64 bit mode—and then that it was not the chip that I had ordered. In the end it eventuated that the motherboard BIOS didn't know about the processor—which of course was an Athlon 64, as the photos show—and thus couldn't set the operating parameters correctly. It seems that Socket 754 chips are pretty much unobtainable nowadays, so in the end we agreed to return the machine and I started again with the same cabinet, an MSI Socket AM2 motherboard and a faster processor (Athlon 64 3500+).
Home to put the new machine together. That wasn't without its surprises either. It worked fine in 64 bit mode, but X didn't load the driver for the on-board video chip (part of the VIA chip set), and the sound driver didn't detect anything. Some research showed that the via driver isn't 64 bit clean. Damn.
Putting in an AGP card wasn't an option, either: this board has PCI Express slots, two of them, but I don't have any PCI Express boards. Finally removed a PCI card from wantadilla and used that. Result: I can now finally display the test HDTV image without dropping frames. Now to investigate the other issues.
Spent most of the day playing around with the new motherboard. Far too much of it isn't supported. Booted back into 32 bit mode and found that I still couldn't load the via driver: it seems that this chip set is too new. I was also unable to find a FreeBSD driver that would recognize the sound chip, which seems to be intimately related to the VGA chip. pciconf -lv reports:
Did get some testing done, which showed that the difference in mplayer performance between 32 bit and 64 bit versions of X was less than the margin of experimental error: so for the moment it doesn't seem worthwhile taking in the disadvantages of 64 bit mode (mainly less driver support).
After that, decided to install Linux, the latest version of Ubuntu (“Dapper Drake”). They've managed to dumb down the installation to a point where it's not possible to set anything except possibly the partition size. How I hate these “modern” GUIs!
As the result of a suggestion by Stephen Rothwell, went looking on the VIA web site and found sources for VGA drivers for the chip set I'm using, released under what is effectively a BSD license. That's in marked contrast to the claims on the web that VIA hasn't understood “Open Source” and provide no support for them. I wonder why the drivers aren't included in the x.org distribution, and whether they're in the XFree86 distribution.
Tried installing the latest version of KnoppMyth. The installation process is no more refined than the previous version which I tried a couple of months ago . The “automatic” install fails with:
Looking at the code, it's in partition_autosize() trying to calculate the disk size based on a variable $SIZE that obviously didn't get set. It's set by the output of fdisk, and it seems to assume that the disk in question is /dev/hda; in this case, for some reason probably related to SATA, it's /dev/hdc. But I want to install, not debug installation procedures.
Tried the custom install and on leaving the Partition-Menu (sic), got
I should be used to this by now, but somehow I still find it incredibly frustrating.
On advice from Ben Herrenschmidt, downloaded the Ubuntu “alternate” CD image, but that didn't give me much more control over the installation. In particular, no choice of what software to install, certainly not whether to start GNOME or not.
The manual disk partition menu (text mode) is marvellous: it has to have a frame, which makes it one line shorter than it should be. It scrolls, but there's nothing to tell you that. Instead, you're left with a menu that looks complete, but is missing the “Next” button: it looks as if there's no way to continue.
Somehow didn't have the courage to try to continue beyond the basic installation. First I need to fix my “new system” scripts to work with Linux as well, then I'll be able to do things much more quickly.
Instead, back looking at the drivers for the VIA chip. What they supply is supposedly tailored for Fedora Core 4 (now happily obsolete), but part of it at least (marked “kernel”) is intended to be incorporated in a build of X.org as a replacement for the via driver. Started that, but ended up with no accessible header files and even less desire to continue, so put it off until next week. I should take a break from computers over the weekend.
Tried Yet Again to install a version of MythTV on the new CVR box, which I'm calling tv2 for the moment. First I started with the version of Ubuntu that I installed last week. I hate Ubuntu! It installs nothing of value, and it doesn't even have a valid apt/sources.list: it appears to have only the CD-ROM, and that doesn't include everything. In particular, I wasn't even able to apt-get Emacs. I also couldn't find a menu item for mounting NFS file systems. To make matters worse, the xterm replacement in the menu misinterprets all Meta-key functions, so I couldn't even type into it. There is an xterm, but I didn't find it in the menus; and like always (it seems) in Linux today, the normal pointers had been replaced by a symbolized hand with truncated fingers. The whole thing looks more and more like a poor copy of Microsoft.
In addition, had more NFS mount hangs. I can't necessarily blame them on Ubuntu, though in this case the lack of correct DNS configuration might have helped. Possibly I should be installing Debian, but there seems to be no way to install the unstable version directly, and nobody I know uses the stable version.
So I went back to installing KnoppMyth again, having first ensured that my disk was in the right place. That worked better, though still not correctly. On reboot, it asks for the root password which I had had to enter, and then rejects it. On further investigation discovered that the password hadn't been set—and that the login in that particular window wouldn't accept an empty password. So ended up logging in on /dev/tty2, setting the password, and returning. The next step is to configure MythTV, which I'll leave until tomorrow.
On with the KnoppMyth configuration. The “automatic installation” asks very few questions: user name and password, “administrator” password, time zone. As I saw yesterday, It doesn't set the administrator password, and today during setup I discovered it doesn't set the user password or the time zone either (though it does create the user name). It also had the same mutilation of IP addresses in the network setup screens that I had noticed (but not reported) earlier (dropped the first digit, so 192 became 92).
Setup is interesting. Here's a detailed description of what I did.
Lots of work on MythTV, and made some progress; most of the information is in the log. It's not clear whether KnoppMyth thinks the installation is complete or not; rebooting the machine still brings the setup menu. This time I didn't kill it quickly enough, and... it killed my configuration! It completely blew away the mythconverg database, and I had to start again! It's a good thing that I've been writing down what I did.
One seemingly unrelated problem I had yesterday was poor picture quality when recording from the new DViCO card. Spent some time playing around and found that it didn't even complete initialization in ceeveear:
Suspecting a newer hardware revision, I compared the PCI parameters. They were identical for both cards, both under Linux and FreeBSD . Linux says:
FreeBSD says:
So do I have a dud card, or is there some undocumented change to the cards that the current drivers can't handle? It would be nice to know what frontend initialization failed means.
While investigating the problem, tried to install FreeBSD on the machine; it failed consistently with disk access problems. Presumably it doesn't understand the new hardware. I've seldom had so many compatibility problems with a motherboard.
On the suggestion of Jan Jones, decided to install MythDora, a Fedora-based MythTV system, on the machine. As usual, finding the documentation is not easy, and managed to download an ancient ISO image (release 2.1; current is 2.32) at a snail's pace (took nearly 3 hours). By that time I had found the correct version and started another download, but it wasn't ready by the evening. The documentation promises a step-by-step pictorial illustration of the installation process, but in fact only gets as far as the end of the standard Fedora installation. I'll have to wait until tomorrow to find out what the installation really looks like.
While waiting for the MythDora download, also tried installing Microsoft “Windows XP Media Center Edition” from the MSDN DVD set. It got about half way and asked for CD 2, whatever that may be. Looks like a bug in the packaging.
Finally burnt an ISO for MythDora and booted it. The results weren't encouraging. It did confirm that the sound hardware is working, the first distribution to do so, but that was about the only good news:
So currently I'm dead in the water. There doesn't seem to be any reason why the X server won't start—on booting it recognized the vesa driver, and this particular image is designed for the VIA chip sets anyway. As I said: not very encouraging.
It's becoming clear that I need to split this problem into hardware and software issues; probably the best is to swap the old and new machine and get the new machine to do the video recordings until I can get MythTV running on the old one, and then address the display hardware issues on the new one.
More work on the black box today, and discovered that there's still a surprising amount of work to do on the build infrastructure. Got that pretty well sorted out, but there's still the can of worms of the startup scripts to look at, and there's plenty to keep me busy there.
In the afternoon, swapped the disks and tuners between ceeveear and tv2 and did a test recording. A good thing too; the “old” card, which works perfectly in ceeveear, produced numerous error messages in the VIA motherboard:
The resultant recording was full of artefacts that roughly paralleled the error messages, so I put the hardware back again and had no further problems.
Somehow I've not had luck with this board (an MSI K9VGM-V with the Via K8M890 chip set):
Looks like time to investigate alternatives. It seems that the nVidia nForce 550 would be a good choice, but I haven't found any in MicroATX form factor.
Spent most of the day looking at DVD authoring software and confirmed my past observations that the coding standard in this area is terrible. Beyond the function name, I have no idea what this code is supposed to do (no comments, of course), but it must be possible to make it more readable:
Doubtless a little documentation of the underlying data structure would work wonders.
Spent most of the day learning about DVD mastering. Pioneer has some relatively useful documentation on the subject, in particular a Technical guide that is unfortunately somewhat out of date and doesn't mention DVD+R at all (only DVD-R). But somehow none of these documents give me the kind of overview I want. The way I see it, there are two different structures to understand:
The DVD is divided into titles. You don't need to investigate the data format to know that, but it's useful to know that the data structures include a single Video Manager Group (VMG) and several Video Title Sets (VTS). The latter represent the individual titles.
The payload data for each of these components is stored in VOB (video object) files, which as far as I can tell are always a specific kind of MPEG-2.
Metadata for each of these components is stored in Video Title Set Info (VTSI) files. They contain things like chapter markers (currently of particular interest to me), subtitles, camera angles and things.
The VMG is divided into the Video Manager Information (VMGI) and the Video Object Set for the VMG menu (VMGM_VOBS). At least, that's what the Pioneer docco says; it looks pretty clumsy to me. Based on my current understanding, these are basically the DVD menus.
The file system structure. DVDs have both UDF and ISO 9660 file systems (they use the term UDF bridge in a manner which is not defined clearly enough). There's a relatively strict hierarchy, since these files need to be readable by DVD players and other systems with low intelligence. There are up to three top-level directories:
Here's an example from a DVD and a DVD+RW, taken from a mount of the DVDs as an ISO 9660 file system under FreeBSD. First, the DVD:
There are a number of things to note here:
DVD+Rs (and presumably also DVD-Rs) also have a directory video_rm:
dr-xr-xr-x 1 root wheel 2048 Sep 4 02:46 video_rm
dr-xr-xr-x 1 root wheel 2048 Sep 4 02:46 video_ts
./video_rm:
total 1
-r-xr-xr-x 1 root wheel 116736 Sep 4 02:46 video_rm.bup
-r-xr-xr-x 1 root wheel 8192 Sep 4 02:46 video_rm.dat
-r-xr-xr-x 1 root wheel 116736 Sep 4 02:46 video_rm.ifo
...
I haven't found out what it's for yet, though clearly it follows the same file naming convention, and from that naming it's presumably global in scope.
Spent the afternoon looking at DVD formats. It's a real can of worms. The following is an attempt to structure it in a way that's easy for me to understand. It may contain errors; please point out any you see.
In the evening, addressed an issue that has annoyed me about mplayer for some time: the on-screen time display bears no relationship to reality when displaying MPEGs (recorded or directly) from digital TV. It seems that the time information embedded in the stream does not get converted, so the “elapsed time” seems to be randomly distributed in the range 0 to 30 hours. Made a change to at least partially fix that—simply subtract the time of the first frame from the elapsed time, and things work almost correctly. That can definitely be improved on, though, so I won't publish yet. In passing I'm once again amazed by how simple it is to make this kind of fix to mplayer, despite a desperate lack of comments. Given that there's no other technical documentation, that's a particular challenge.
=== root@teevee (/dev/ttypb) ~ color="blue">31 -> freevo setup --geometry=1280x720 --tv=pal --chanlist=australia
Wow! That makes it pretty useless. I suppose I need to go back and try my luck with MythTV again. Or maybe I should learn python and fix it. Continued anyway and was able to point it at my Maybe directory (for stuff that I may look at if nothing else comes up), and found:
That scared me—did it maybe first delete the contents? No, they're still there:
=== root@teevee (/dev/ttypb) color="red">/spool/Images 43 -> l Maybe/
The problem is, of course, that the thing insists on “extensions”. Once given that, it works, but I still can't display full screen.
Spent most of today looking at remote controls under FreeBSD. I have this remote control that came with the DVICO tuner card; it's unclear what the real name is: the unit itself bears the name “FusionREMOTE”, but the Linux dmesg output states:
It's also not clear which letters are written in capitals and which in lower case; I've also seen “DViCO” and possibly others.
I've already had considerable pain trying to get lirc, the Linux Infrared Remote Control software, to work. There's a partial port for it in the FreeBSD Ports Collection, so decided to play around with that. Discovered that my particular remote control uses the HID (Human Interface Device) interface, called /dev/usb/hiddev in Linux. The corresponding FreeBSD device is /dev/uhid. Played around with uhidctl(1) and added a hex dump feature that showed that repeated data sets the high-order bit of the (16 bit) data word:
The data is delivered in bits 1 to 7 from the left; the last 8 bits are always 0xfe. Looking at the data in lircd.conf, this corresponds pretty well. Clearly this was the down button:
What about the kernel API? It's described in uhid(4) for FreeBSD. Linux doesn't seem to have kernel man pages; instead, the interfaces are described in the directory /usr/src/linux/Documentation, in this case in /usr/src/linux/Documentation/usb/hiddev.txt. The interface is very different and I spent some time looking at the LIRC code in daemon/hw_hiddev.c, where I found obscenities like:
Looking at the interface API, this probably does represent an error, but where's the analysis? Is the value returned greater than, equal to or less than 0? Where's errno when you need it?
In addition, this code seems to pass parameters via global variables:
Round about here, I got sick of even looking. lirc has proven to be a real pain at the best of times. Do I ignore the problem, fix it or write a different interface? Spent some time thinking about that and came to the conclusion that I had to live with the problem, at least in the short term, but boy, is that painful! This code also showed something that could be a problem: Linux refers a struct hiddev_event with the members hid and value. value is clearly the value displayed by usbhidctl, but I don't see anything corresponding to the hid value of 0x10046. Where does that come from?
More work on remote controls today, and made reasonable progress. The more I look at lirc, the less I like it. Starting the daemon, got the message
Why? Went through the code and checked: it seems that the directory for the socket didn't exist. How much easier it is with this patch:
Now we get:
Is that really so much work? How many people have scratched their heads about this one?
Huh? Why can't it open /dev/uhid? Why did it “catch a signal”? With some patches:
it became blindingly obvious:
Why couldn't that be put in as well?
Presumably the “caught a signal” is lircd's own way of stopping (or committing suicide). I didn't bother to fix that one.
In the process of testing this code, discovered another problem, not related to the software: the DVICO remote control suffers from premature repetition. It's almost impossible to press a button without repeating, and presumably it would blow the mind of the designer of my brain-dead Motorola L6 mobile phone: during the time you have to hold down the “off” button to turn on the phone, the remote control repeats 27 times. This isn't a FreeBSD-related problem, nor an isolated case: it happens on both of the controls I have, and on both FreeBSD and Linux. Ended up writing code to ignore the first two repeats, which still makes it very responsive.
Next was to actually use the functionality with mplayer. That required going back to the inaccurate, incorrect, incomplete and contradictory LIRC documentation. It seems I needed a .lircrc file. Format? Who knows? I found multiple examples, but no documentation. One of the examples was the DVICO/LIRC HOWTO, which included a file which I copied as discussed. No reaction. Why? Well, an obvious reason is that the button names mentioned in that file don't correspond to the button names in the /etc/lircd.conf file. The format appears to be straightforward, but who knows what gremlins lurk? Spent some time looking at that. The format of the file appears to be (caution: once again this is an example, not the (missing) documentation):
Here the names on the right of the = sign are clearly strings. Right is presumably intended to the name of the “right” button (called right in my lircd.conf), mplayer the name of the program, pt_step 1 an entry in the ~/.mplayer/input.conf file, and the parameter to repeat might mean that further repeats of this button should be ignored (though in this case that seems a bad choice).
This clearly gives me the option of controlling multiple programs from the same remote control, a very useful function. But it also seems to imply that specific buttons relate to specific programs, which is far less useful. The top of the control has four buttons marked “DTV”, “MP3”, “DVD” and “CPF” (whatever that may mean). Clearly the intention is to divert the functionality of the other buttons to one of four programs. How do you do that? I don't know yet.
In the afternoon, back to the remote control stuff, and found yesterday's assumptions confirmed, including the concerns: mplayer reacts to the remote control even when it's iconified, so if you have multiple mplayers running (something I do frequently), presumably all will react to the remote controls. I need to look at doing something there. Also had permission problems with lirc. By default, the socket is writable by root only:
=== grog@teevee (/dev/ttyp0) ~ color="blue">4 -> ls -l /var/lirc/
It's not clear why mplayer needs to open this file with write permissions, but it stops it from working. I also don't know wat /var/lirc/lircm is. Clearly it's been there for a while, but I didn't mention any work on lirc on 23 August.
More work on mplayer today, still trying to get the on-screen display showing correct values. This time I was concentrating on getting the total time correct; frequently it would show values less than half the correct value. I had suspected some kind of overflow, but after tracing through some particularly emetic code, found that the input stream was either supplying incorrect bit rate data, or was being interpreted incorrectly. After that, found that it also gets reported accordingly on mplayer startup. The values seem to have no relationship to the real information; do they come from the stream, or is mplayer misinterpreting them? Here's what I get for the Adelaide TV channels:
There seems to be no relationship between the bit rate and the resolution, and in addition some “HD” streams are nothing of the sort. In many cases, the HDTV streams have a slower specified bit rate than the normal resolution. Seven HDTV is a special case: it seems to be very weak, and frequently enough I get nothing at all.
What does this mean for mplayer? Clearly it can't always relate on the specified or calculated bit rate. I think the best choice would be to look at the file size and calculate from there. That would also help where the timer wraps around in the middle.
Also did some work on our project; in this case, mplayer is reading from a pipe, and we somehow need to find a way to get it to flush the pipe when it starts so that it's showing current data and not the data that had collected in the pipe. That's straightforward enough: a keyboard input would do it, but in our case it looks like we'll send the command down another pipe:
=== grog@ceeveear (/dev/pts/2) ~ color="blue">121 -> mkfifo /var/tmp/mplayer-command
=== grog@ceeveear (/dev/pts/3) ~ color="blue">36 -> mplayer /dev/dvb/adapter0/dvr0 -really-quiet \
=== grog@ceeveear (/dev/pts/2) ~ color="blue">122 -> echo "seek 10" > /var/tmp/mplayer-command
This example was done on Linux, but it works exactly the same way under FreeBSD. Note the two terminals: mplayer writes to the terminal.
More work on mplayer status lines. Calculating the bit rate isn't nearly as easy as I expected it to be.
How do you get mplayer to catch up with an input stream? You can create a command pipe with input file=/pipename, but if you send a seek beyond the end of the current input stream, it hangs for a while and then crashes. In our prototype, that then brings down the entire application. Spent some time trying various options, but didn't find a solution.
Spent most of the day looking at the cxm driver for the Hauppauge PVR250. It could do with a lot of improvement; I've already updated the man page for the companion pvr-setchannel program, but there's much more to look at. How do resolution and data rate compare? Checked some DVDs and discovered that a commercial DVD (or at least the one I tested) runs with a resolution of 720x576 (PAL) with 25 fps and 9.0 Mb/s. By contrast, data recorded from TV with my impossible Digitrex GKX-9000 DVD recorder at “standard” resolution (2 hours, 7 minutes on a single-layer 4.5 GB DVD+RW) had a resolution of 352x576,) 25 frames per second and 9.396 Mb/s.
Part of our requirements are also to put 2 hours onto a standard DVD. We're (currently) doing it at full 720x576 resolution, but are dropping the bit rate to 4.52 Mb/s. That works, but is it better than half resolution and full data rate? The way we're doing it also requires patching the driver, and what I'm looking at at the moment is to specify the data rate separately from the “profile” (which is hidden behind screen resolutions and implicit standards).
Started importing the cxm driver for the Hauppauge PVR-250 into the FreeBSD kernel tree; it's currently in the Ports Collection. No great surprises, but enough work.
Spent too much time trying to build a kernel with the cxm driver. Building the module works fine, but the sources require a header file iicbb_if.h that is (apparently) only built when the modules are built. I wish I understood the build process better.
In the evening, tried to look at an MPEG-2 stream recorded on ABC “High Definition” TV at 1280x720 non-interlaced (“progressive”).
For some reason, it flat-lined the CPU and couldn't display properly. By contrast, the other night I had watched “9 HD”, with the rather adventurous resolution of 1440x1088 and more than 50% higher data rate:
On that occasion, it had used up about 75% of the CPU. Trying again today, it flat-lined the CPU, though it displayed better than ABC.
What's going on here? There are a number of things that need clarification:
In summary, I'm left wondering whether there's something else involved, possibly the scheduler or elsewhere in the kernel. More investigation needed.
For some time I've been meaning to investigate software for creating DVDs from broadcast MPEG streams, and I've been putting it off because I feared the amount of work it would take to put it right. That's all the funnier because that's almost exactly what we do with our Black Box project, but maybe the pain we've had there explains some of the fear.
Today was the day; went through the Ports Collection multimedia section once again and came out with a bewildering number of programs, all described in enormous fixed-width fonts. How I hate this broken web site! In the end, asked on IRC, where Jürgen Lock was very helpful in describing how he does it. First, though, I tried replex.
Trying that gave the following output on the screen:
=== grog@teevee (/dev/ttyp4) color="red">/spool/Images/Already color="blue">8 -> replex -t DVD -o 3suns.mpg tre-solar
They were matched by the following output on the console, which indicates that it's trying to write to a DVD. There's nothing in the “documentation” that explains why it's trying to do this.
Finally, replex died with the message:
Possibly I'm doing something wrong, but without any documentation, like so much multimedia stuff, it's pretty much useless.
Jürgen's recommendation is Project X to disassemble the file into its component streams, then dvdstyler to put them back together again. That doesn't seem to make sense: the VOB files are MPEG-2 program streams, and the input files are MPEG-2 transport streams, which are related, so it seems unnecessary to take them apart and put them back together again; replex doesn't need that. On the other hand, replex also doesn't seem to work. Also, dvdstyler is really a program for creating menus for a DVD, something that I don't want. But can you create a DVD without a menu? I don't know. So I'm guessing that Jürgen's method is one “that works” rather than an optimal procedure.
http://www.FreeBSDFoundation.org/cgi-bin/download?download=diablo-caffe-freebsd6-i386-1.5
.0_07-b01.tar.bz2
with a web browser and "Accept" the End User License Agreement for
"Caffe Diablo 1.5.0". Please place the downloaded
diablo-caffe-freebsd6-i386-1.5.0_07-b01.tar.bz2 in /usr/ports/distfiles.
.*** Error code 1
After doing that, it installed, but there was no documentation. The initial screen looks pretty, but is completely confusing:
I was about to give up on it until Jürgen came along and helped me through it, step by step. In the past I've always refused such offers of help: they shouldn't be necessary. But, sad as it seems, they are necessary, because people write programs without any documentation. Basically, the procedure is:
Project X does remember some window size settings, notably for the home window, where additional size makes no difference. Here you can enlarge the window and show more files, but it forgets the settings and opens the same small window the next time around. About the only thing you can do to improve things is to select the Babylonic cuneiform icon at top right, which changes the display to something containing information, including dates written the wrong way round:
If the text weren't so tiny, it would be quite helpful. Clearly there's a lot of stuff to play with here.
In my case, though, the results were completely different: I hadn't selected demux in the Action: bar at the top, and by default it's set to PIDFilter (1:1 Copy). This is another thing that gets remembered, so it's understandable that Jürgen had forgotten it. After selecting that, and restarting, it created a number of new files:
To be fair to the program, I didn't read the documentation, since Jürgen was navigating. The first thing you'd expect to do with the initial screen would be to select something from the Files tab:
In fact, that's just for loading project files (and has this same stupid, brain-dead ignorance of the working directory that all “modern” programs seem to have nowadays). To get at your files, use the Directories tab on the side. Then drag the files shown on the right in specific sequence:
At the end, it should look something like this:
Next, create a menu:
Then right click on the arrow image and check the Action. In a simple case like this, it's probably what you want:
Finally, use the File–>burn dvd menu option to burn the DVD. Again, dvdstyler shows a lack of understanding of directories by trying to build the DVD image in ~/dvd. You can change the name, though, and this time it even lets you type in a path name:
This image can be over 9 GB in size, so that's a non-trivial problem.
This is done with mplex, which is not a ball of fire. In this example it took several hours to create the image. That's not all that dvdstyler does, but it takes up the majority of the processing time.
Spent a lot of time today writing up yesterday's experiences with DVD mastering, and continued investigating the reasons why some of the resultant DVDs wouldn't work in my brain-dead Digitrex DVD recorder, though mplayer had no problems with it. With the help of mplayer, established that there were format differences:
The first was Astérix DVD that I burnt was “progressive scan” (i.e. non-interlaced) and “aspect 3” (i.e. 16x9 aspect ratio), and the second, the one that did work, was interlaced with “aspect 2” (i.e. 4x3). It also had a different resolution: on closer examination, none of the resolutions are 4x3. 720x576 is 5x4, while 704x576 is 11x9. To make matters worse, Project X offered different ratios again: 0.6735 (2000x1437), which it calls “4:3”, and 0.7031 (really 10000x7031), which it calls “16:9”. Clearly there's something to be learnt here. It also uses the term DAR, which apparently means Display Aspect Ratio. What other kind of aspect ratio is there? I suppose I might find out if I can make sense of the ratios I've seen. Tried a small test after setting the horizontal resolution and aspect ratio to match the image that did work. This is the top right of the Project X screen after the changes:
Indeed, the result worked. Which change was it? My guess is the aspect ratio. Now to learn how to chop off leaders and trailers. The image contained nearly 400 MB of junk at the beginning because the TV stations never start broadcasting on time. I thought I had cut it off, but it ended up on the DVD anyway.
Did some more work on DVD mastering, and confirmed yesterday's suspicion that it was the aspect ratio that caused the Digitrex DVD recorder to refuse previous DVDs. But this is still so slow! Read up on the quite considerable documentation in the Transcode wiki, discovering in the process a script called any2DVD, which may have the distinction of being the largest shell script I have ever seen:
Set to porting that—it doesn't have any installation environment, so I had to do it manually. Wasn't finished by the evening.
Mail from James Andrewartha telling me that DVD pixels are not square, thus explaining the strange aspect ratios I had noticed yesterday. What a can of worms!
Still more DVD authoring stuff. Fixed the Makefile for any2DVD, maybe a little early: despite being a shell script, it had a number of surprises: it claims to run with /bin/sh (it doesn't, but in Linux /bin/sh is a link to /bin/bash, with which it does run); it uses a flavour of sed incompatible with BSD sed; and it makes this horrible assumption that the terminal background is black, then produces illegible messages:
|
|
||||||||
In the evening, our Digitrex DVD recorder gave up the ghost in a slightly different way: the front panel on/off switch got stuck in the housing and wouldn't come out:
It's not clear what caused it to work in the first place; possibly a coil spring that has since got lost. In any case, I can't complain. It's by far the worst piece of electronics I've bought in the last few years, but since the manufacturer refunded the purchase price and didn't want it back, I suppose I've got my money's worth. Took it apart and investigated, taking some photos in the process. There might be something salvageable in there (at least the DVD+RW drive). Put it back together without the button; now we'll have to use a cotton stick to locate the real switch on the PCB board behind the front panel on those occasions where we still use it.
Spent a lot of the day looking at any2DVD, gradually coming to the conclusion that I had committed it too hastily. Shell scripts are a pain to debug at the best of times, and when they're this size, it's hell. I seem to have left behind a trail of half-completed ports, so this might be one that I should just forget.
For some time I've been worrying about displaying HDTV on teevee.lemis.com, an AMD Athlon XP 2500+ (1833 MHz) running FreeBSD 6.1. Basically, earlier this month I had established that, try as I might, I couldn't get the CPU grunt to render 720p effectively with any driver available to me. Earlier, in August I had noted:
The nVidia driver does not improve performance; in fact, it decreases it.But what about hardware MPEG rendition? Did some investigation and came up with a reference on mythtv.org. Installing the nVidia drivers isn't enough: you also need to use XvMC, X video motion compensation, which you invoke with:
The -vo specifies the video driver, and -vc specifies the video codec.
Installed the latest nVidia driver (/usr/ports/x11/nvidia-driver) and the configuration utility (/usr/ports/x11/nvidia-xconfig) and ran the latter, which did nothing useful apart from introduce gratuitous changes to the xorg.conf file. The real diffs are:
Section "Module"
Load "dbe"
- Load "dri"
Load "extmod"
Load "glx"
Load "record"
@@ -119,7 +119,7 @@
#Option "FPScale" # [<bool>]
#Option "FPTweak" # <i>
Identifier "Control"
- Driver "nv"
+ Driver "nvidia"
VendorName "nVidia Corporation"
BoardName "GeForce 6200"
BusID "PCI:1:0:0"
I don't know why the config program removed dri; possibly it doesn't support it. I suppose I should try replacing it and seeing what happens.
With those changes, mplayer worked much better. Instead of dropping frames like crazy at 100% CPU usage, it worked fine with 75% idle. top shows that the 25% was all used by mplayer; X didn't even register. So it looks as if I can stick with X.
| Greg's home page | Greg's diary | Greg's photos | Copyright |
