PRISMIQ MediaManager for Linux Technical Information

This documentation is up to date as of December, 2003, coinciding with the release of version 4.0 of PRISMIQ MediaManager for Windows Computers. If there are any errors or omissions, please email us at linux-preview@PRISMIQ.com! (Note, the linux-preview address is for both the Mac OS X and Linux releases)

Technical Details Table of Contents

• Example XML Datafiles
• Notes on media.map and media.xml
• Playlist information
• Supported File Determination
  1. Summary
  2. Connection
  3. Communication syntax
  4. Queries and responses
     1. Query Examples
• Development Server Process Information
  1. The ScreenServer, a Tomcat server
  2. The Video On Demand Server (VOD), "mmserv"
  3. The default PRISMIQ Transcoder plugin, "prismiq_xcode"
• Multicast Routing information
• Example Debugging Process

Example XML Datafiles:
The Linux Preview Edition of the MediaServer contains sample xml files that can be used as a guide. Download the Linux Preview Edition of the MediaServer here: http://www.prismiq.org/modules.php?op=modload&name=Downloads&file=index&req=getit&lid=2. The sample files are located in the "sample" subdirectory of the "docs" directory that is at the top level of the install. Each sample file contains comments that describe the individual file, as well as the various entries in the file. Note - there were two minor additions to the xml files in the version 3.2 release, so documentation from the 3.1 release is slightly out of date.

New to Version 3.2:

playlist.xml: Each playlist will have the addition of two extra fields:

<sortcolumn>title</sortcolumn>
<sortdirection>Ascending</sortdirection>

If the value is not defined for a given playlist, this field can be blank:

<sortcolumn/>

Or it can be omitted. The User Interface will not display any sorting information or maintain sorts if these values are correct, but will save the values if a user chooses them.

userprefs.xml: The "location" item will support both locations in the USA, as well as certain locations in Canada. The location item will contain a new field, country, such as:

<location>
<country>Canada</country>
<zipcode>H3Z 3B8</zipcode>
<city>Montr&#xe9;al</city>
<state>Qu&#xe9;bec</state>
</location>

Note that the accented e has been encoded - all non ASCII characters should be encoded using the form "&#x%x", where the %x character is the hexadecimal value of the character.

An Example of a city from the US:

<location>
<country>United States of America</country>
<zipcode>93101</zipcode>
<city>Santa Barbara</city>
<state>California</state>
</location>

More notes on the location item:

- The country can be either "Canada" or "United States of America".
- Arbitrary cities can be used for the United States, as the zipcode is used for the data retrieval.
- Canadian cities must be taken from a list of supported cities, which will be published.

Notes on media.map and media.xml:
As you can guess from the name, media.map and media.xml store data about all of the media that the MediaPlayer can access. For detailed information, see the example files contained in the MediaServer Preview, but there are a few additional notes:

1) Do not hold file handles open to the files, as this may block access to the file.
2) media.map and media.mxl must have matching information. The "id" field in media.xml will be mapped to the [id] field in media.map when the VOD server attempts to stream the media to the MediaPlayer
3) The "type" field in media.xml is reasonably arbitrary, and merely identifies the file type. Typically, this is simply the file extension.

Playlist Information:
Playlists are allowed to be only a single type, i.e. audio, video, or image (the actual type namefor images is "photo" for historical reasons). The list of all items in a given type, i.e. all video, is generated on the fly from media.xml. Playlist names can have almost any character in its name with the exception of the percent signal (%) and the question mark (?).

A user interface should enforce that playlists of a given type only contain media of that type. Any audio only file should show up only in an audio playlist, and any media with video content should obviously go in a video playlist. In the case where a video has a supported audio codec, but an unsupported video codec, it's up to the UI designer to decide whether the file should be added. A file of this type will appear as a black screen, with audio playing. In order to determine if a given file is supported, see the next section, "Supported File Determination".

Image playlists can contain Jpegs, as well as slideshow data such as display time for each slide (or no time entry for manual advance), as well as an attached playlist or audio file to play when showing the image on the MediaPlayer. For more infomation, see the sample data files.

Supported File Determination:
Since the PRISMIQ MediaServer can have user written plugins for arbitrary content types (see http://update.prismiq.com/plugins/ for more info), there is not a static list of file types supported. Also, given that some media files are "container" formats, such as AVI, not all files of a give type will be supported. An example of this would be an AVI file that has it's video encoded in Sorenson, and it's audio encoded via mp3 - the default plugin for the MediaServer will be able to read the mp3 audio track, but not the Sorenson video track. A RealMedia file will not be readable at all. Since the User Interface really should not contain logic for figuring out what codecs are supported, but the User Interface does need to correctly discover media, the MediaServer has a text based socket protocol that can be used to query for a list of supported file extensions, as well as test individual files for compatibility with all running plugins. What follows below is as specification document detailing the communication.

1. Summary
2. Connection
3. Communication syntax
4. Queries and responses
   1. Query Examples

1. Summary:
The PRISMIQ family of software has two programs that need to exchange information on supported media files, the user interface, here after referred to as "MediaManager" (MM) and the Video On Demand server (VOD). On Linux, the VOD server is named "mmserv", and on Windows, it is named "WinVOD.exe".

This document describes the method that MM uses to send a query to VOD
to learn about a media file and whether VOD can use it.

2 Connection:
Media Manager connects to the VOD server with a "localhost" (127.0.0.1) TCP socket on port 554. The VOD server listens on port 554, and if the client is local (127.0.0.1), it sets up a connection for validating local files, otherwise it does the RTSP connection with the client (player).

The VOD server may handle multiple simultaneous TCP file query sessions.

A file query client should keep the connection open as long as needed to handle all its file requests, rather than opening and closing a new connection for each query. The client may close the connection any time, at its convenience.

3 Communication syntax:
The two applications shall communicate with ASCII lines, terminated by a standard newline character (decimal value 10). Each side should tolerate the carriage return character (decimal value 13) from the other.

Commands, result codes, and other well-known tokens shall be case insensitive. Other parameters may require mixed case (filenames for instance).

White space shall separate tokens, except inside double quotes. Any string within double quotes shall become a single token, and the quotes discarded. The double-quote, asterisk, and non-printable characters may be passed as a parameter in certain circumstances, and must be translated into an escape sequence consisting of the asterisk and two hexadecimal digits. This is of particular interest for Macintosh filename support and other languages (eg, Chinese).

All commands (from MM to VOD) shall consist of a command token followed by zero or more parameters. All responses (from VOD to MM) shall consist of a numerical code followed by zero or more informational items. The numerical codes shall be organized into categories so that MM can guess the meanings of unknown codes.

A command from MM may have a synchronization number immediately preceding the command token. This number shall be enclosed in brackets like this: [1234] and the response for that command shall include this same string immediately preceding the numerical result code. If MM needs to use the bracket characters as a command parameter, it must specify a synchronization number. The synchronization numbers are solely for the benefit of the MM, and VOD must not assume that the numbers will be sequential or follow any pattern. The MM program may encode its own state, flags, or other information into this number if desired. The numbers must be in the unsigned range from one to 2^31-1, and zero is reserved.

MM may use synchronization numbers to send multiple queries to VOD in bulk, and harvest the result codes as they arrive. This might reduce the command latency considerably.

4 Queries and responses:

Queries:

Response code examples:

4.1 Query examples:

Example #1, Media Manager queries about two files:

Example #2, Media Manager sends multiple queries in batch:

[0001] GET_STATUS "c:\media\movie1.avi"
[0002] GET_STATUS "c:\media\movie2.avi"
[0003] GET_STATUS "c:\media\movie3.avi"
[0004] GET_STATUS "c:\media\movie4.avi"
[0005] GET_STATUS "c:\media\movie5.avi"
[0006] GET_STATUS "c:\media\movie6.avi"

[1] 200 OK "A/V: MP3, MSMPEG4V3"
[2] 200 OK "A/V: AC3, H263"
[3] 200 OK "A/V: WMA2, MSMPEG4V2"
[4] 415 Unsupported Media Type
[5] 200 OK "Audio: MP3"
[6] 200 OK "A/V: MP3, ?" (audio supported, but not video)

Note that the responses drop the leading zeroes since they are
treated as integers.

Example #3, Filename with spaces and double quotes (my "home" movies)

[10] GET_STATUS "c:\media\my *22home*22 movies"
[10] 200 OK "A/V: MP2, MPEG4"

Example #4, get a list of supported file extensions:

Example #5, get file extension strings:

Development Server Process Information:
If you want to customize the starting and stopping of the MediaServer for Linux, there are (at least) three processes that need to be started. The included start.sh and stop.sh scripts will perform the actions for you, but in case the community wants to create their own method for starting and stopping the processes, the following information is included.

1. The ScreenServer
The ScreenServer is responsible for communicating data and user interface information to the MediaPlayer. It is a customized Tomcat Server (version 4.0.3), and requires at least Java 1.4.1. It does support most Tomcat functionality. The only requirements are to set the correct classpath, and execute the main class. The Classpath is somewhat long, and is set correctly in the script named bin/functions.sh. Starting the Tomcat Server is simple - from the bin directory, just execute:

$JAVA_HOME/bin/java -Djava.awt.headless=true -DWEBAPPS.HOME=$WEBAPP_HOME Tomcat.EmbeddedTomcat

This will output logging information to standard out, so for speed, it's recommended to redirect this output to /dev/null. Also, the "-Djava.awt.headless" option will ensure that no errors are generated if the MediaServer is running on a machine that does not have a windowing system enabled, or with permissions for the owner of the Java process to utilize the windowing enviroment. Port settings on the ScreenServer are set via the "screenserver.properties" file, located in mserver_4101/bin/gui/. The default properties file is shown below:

#PRISMIQ ScreenServer Settings File
#Wed Dec 03 10:00:59 PST 2003
com.prismiq.screenserver.port.manual.warn=true
com.prismiq.screenserver.port.current=8081
com.prismiq.screenserver.port.autoselect=true
com.prismiq.screenserver.port.manual.number=8081

In this file, the line "com.prismiq.screenserver.port.current" indicates that the ScreenServer is currently operating on port 8081 (this value will be overwritten at runtime with the current port number), and that the ScreenServer will attempt to discover it's own port upon startup (com.prismiq.screenserver.port.autoselect=true). If "autoselect" is set to "false", then the manual port number will be used (com.prismiq.screenserver.port.manual.number=8081). The first line, "...manual.warn" will only function if the server is not running headless, which is not the case by default.

There is currently one more supported option in screenserver.properties - "com.prismiq.psap.validinterfaces". This entry controls which network interfaces the ScreenServer will attempt to send announcement packets on. If this like is blank, or not present, the MediaServer will send announcment packets on all available (non-loopback) interfaces. If there are valid interface names on the line, such as in the following example, then the announcements will only be sent over the named interfaces.

com.prismiq.psap.validinterfaces=eth1,eth2

To stop the ScreenServer, simply open a socket connection to port 8005. This will cause the server to clean up and exit.

If the User Interface changes datafiles, the ScreenServer can be triggered to reload any changed information by opening a socket connection to port 8006.

2. The Video On Demand (VOD) Server
The VOD server is fairly self contained, and simply needs to be launched from the bin/gui directory:

../mmserv -d

Due to the fact that the VOD server needs to bind to the RSTP port, port 554, it is setuid to root. Also, you should ensure that the tag directory exists and is readable and writable.

For diagnostic purpose, the user can telnet into the server on port 554 from another machine (localhost will NOT work). To do this, telnet in, type "login" and press ENTER twice. Then enter the password. The default password (set in mmserv.conf) is "x88rx2c2ei". To change the password, or add more, you can log in and use the "hash" command to generate a new password hash. Cut and paste this hash into the mmserv.conf file, and use the "reload" command (from mmserv CLI) to activate the changes. Or kill the server ("killall mmserv") and restart it.

Note that after killing the server, it may take up to a minute for all the RTSP sessions to die out. During this period, the server CANNOT be restarted, since it cannot bind to port 554 until these old sockets die out.

3. The Default PRISMIQ Transcoder Plugin, "prismiq_xcode"
Simply launch prismiq_xcode in the background. To kill it, simply send a SIGTERM.

Multicast Routing Information:
The PRISMIQ MediaServer and MediaPlayer use multicast packets to find each other on the network. If your machine running the MediaServer has multiple network interfaces, the multicast packets might get routed over the wrong interface, which will cause the MediaPlayer to not be able to find the MediaServer. Some indications of this include:

• Always being disconnected, or gett "Can't Find MediaServer" messages on the MediaPlayer
• Being able to browse media on the MediaPlayer, but when attempting to play a media file, receiving unreachable messages.

If you experience this, the problem is most likely that your machine is routing multicast packets over the wrong interface, and you need to change the routing table. The method to do this varies among varions operating systems and versions, so check Google.com or your Documentation for more info.

MikeM42 from the PRISMIQ.org forums contributed some great info on Multicast Routing and the MediaPlayer:

So, the problem is that on a linux box with multiple interfaces, particularly those configured as a router/firewall, the default routing rules for multicast packets send them to the external network instead of to the usually preferred internal network. The solution is to provide a routing rule for multicast packets. This should be put with your other routing rules, for me, I put them in /etc/rc.d/rc.local

Code:
/sbin/ip route add 224.0.0.0/4 dev eth0

Keep in mind that i'm using an Ipchains-style routing package on a RedHat 8 system, and that eth0 is connected to my internal network, where the prismiq is located.

For more information, a good resource is http://www.linuxguruz.com/iptables/howto2.4routing-8.html.

Example Debugging Process:

If you can't connect, there's a few different things to try. In any case, the PRISMIQ.org Community Forums can be a good place to get help - http://www.PRISMIQ.org. Here are some basic debugging steps to try out, and narrow down the problem.

The first thing to check are file permissions and ownership on the file "bin/mmserv" This file needs to be owned by root, and setuid. If you can see port 554 or "rstp" when running the command "netstat", then this is likely fine. To verify, run the following command:

[kerry@apu ~/mserver_4101]$ ls -l bin/mmserv
-rwsrwsr-x 1 root kerry 404453 Dec 1 15:15 bin/mmserv

You want to see that the file is owned by root, with a group of whatever user installed the binary, with the file permissions as above. If this is not the case, then fix the file:

[kerry@apu mserver_4101]$ sudo chown root bin/mmserv
[kerry@apu mserver_4101]$ sudo chmod +s bin/mmserv
[kerry@apu ~/mserver_4101]$ ls -l bin/mmserv
-rwsrwsr-x 1 root kerry 404453 Dec 1 15:15 bin/mmserv

(also, when checking for network ports, run "lsof -i", that shows only socket activity, whereas the plain lsof shows all file activity, as well)

If that works, try connected to the MediaServer:

[kerry@apu mserver_4101]$ telnet localhost 554
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
exit
Connection closed by foreign host.

If it just hangs at the "Trying 127.0.0.1...", then it's not up and listening, and is likely not owned by root.

If mmserv is listening, then it is likely that the MediaServer is ok, and it's time to move to the ScreenServer, a Tomcat server.

First, see if if there are any tomcat processes running:

[kerry@apu ~/mserver_4101]$ ps auexww | grep java | grep EmbeddedTomcat | grep -v grep | head -2
kerry 19767 5.7 6.0 228320 31344 pts/1 S 15:44 0:01 /usr/java/bin/java -DWEBAPPS.HOME=/home/kerry/mserver_4101/bin Tomcat.EmbeddedTomcat
kerry 19792 0.0 6.0 228320 31344 pts/1 S 15:44 0:00 /usr/java/bin/java -DWEBAPPS.HOME=/home/kerry/mserver_4101/bin Tomcat.EmbeddedTomcat

(that's a somewhat involved command line, but it looks for everything, and shows as much command line as possible. We just care about the ones running Tomcat.EmbeddedTomcat, and just to make sure there's a few of them running (there should be about 10 of them)).

If you don't see these java processes, then Tomcat didn't start, and the MediaPlayer won't be able to get data from the PC.

Make sure that you have Java 1.4.1 or 1.4.2 installed, and that you have set $JAVA_HOME either in your .cshrc/.profile/.bashrc, or in .mserverrc in the mserver_4101 directory.

If you do have everything set up what seems to be right, let's ask Tomcat for some status info:

[kerry@apu ~/mserver_4101]$ cat bin/gui/screenserver.properties | grep current
com.prismiq.screenserver.port.current=8081
[kerry@apu ~/mserver_4101]$ wget http://localhost:8081/mserver/wilson/status.txt <==== Note the use of the port from the line above!
--15:49:03-- http://localhost:8081/mserver/wilson/status.txt
=> `status.txt'
Resolving localhost... done.
Connecting to localhost[127.0.0.1]:8081... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-shockwave-flash]

[ <=> ] 162 158.20K/s

15:49:03 (158.20 KB/s) - `status.txt' saved [162]

[kerry@apu ~/mserver_4101]$ cat status.txt
Server: MGuide 1.18
Server-Built: Dec 1 2003 15:51:36
Document: status.txt
Date: Mon, 01 Dec 2003 15:52:40 -0800
Working-Directory: /home/kerry/mserver_4101/bin

This is what you should see, this means that the ScreenServer is up and running normally. If you see something similar to the following, the ScreenServer is not starting correctly:

[kerry@apu ~/mserver_4101]$ wget http://localhost:8081/mserver/wilson/status.txt
--15:55:15-- http://localhost:8081/mserver/wilson/status.txt
=> `status.txt'
Resolving localhost... done.
Connecting to localhost[127.0.0.1]:8081... failed: Connection refused.

If this is the case, make sure that your Java Environment is set up correctly. You can also edit the value of TOMCAT_LOG on line 62 in bin/functions.sh to point to a real file rather than /dev/null - this will output logging data from the ScreenServer that may help determine the cause.

(Return to top)

Appendix 1 - Example Hash function used by MediaManager for creating media IDs.:

The Following is a sample of the hashing function used to create the media id's in media.xml and media.map.

#include <stdio.h>
/*****************************************************************************
 *  Copyright (C) 2003 PRISMIQ, inc.  All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *      notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *      notice, this list of conditions and the following disclaimer in the
 *      documentation and/or other materials provided with the distribution.
 * 3. The name of the author may not be used to endorse or promote products
 *      derived from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
 * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
 * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
 * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
 * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 *
 *****************************************************************************/

/******************************************************************************
 * The Hashfunction used by PRISMIQ MediaManager to generate Unique ID's for
 * media items. The function is called for media items with a string that is
 * the concatenation of the full file name and path (as stored in "filename0")
 * with the "file type", that is merely the file extension.
 * 
 * For example, for the file:
 *     "C:\TestContent\Henry Mancini - The Pink Panther Theme.mp3"
 * The hash function would be called with:
 *   hash("C:\TestContent\Henry Mancini - The Pink Panther Theme.mp3mp3");
 * and gives a hash value of:
 *   4249900584
 *
 * The hash function below builds on the ideas from the hash function posted to
 * Usenet by Chris Torek, October 1990.
 *****************************************************************************/
char * hash( char * hashstr )
{
    const void* key = (void*)hashstr;
    int len = strlen(hashstr);

    /* Build upon Chris Torek's hash function */
    unsigned long h, loop;
    unsigned char *k;

    char * retval;

#define HASH4   h = (h << 5) + h + *k++;
    h = 0;
    k = (unsigned char *)key;
    if (len > 0)
    {
        loop = (len + 8 - 1) >> 3;
        switch (len & (8 - 1))
        {
            case 0:
                do
                {        /* All fall throughs */
                    HASH4;
            case 7:
                    HASH4;
            case 6:
                    HASH4;
            case 5:
                    HASH4;
            case 4:
                    HASH4;
            case 3:
                    HASH4;
            case 2:
                    HASH4;
            case 1:
                    HASH4;
                } while (--loop);
        }
    } 
    retval = (char *) malloc(32);
    if (!retval) {
	perror("malloc");
        return NULL;
    }
    sprintf(retval, "%u", h);
    return retval;
}

/****************************************************************************
 * 
 * Sample main function to call hash function with. Will hash each command
 * line argument.
 *
 ****************************************************************************/
int main (int argc, char ** argv) {
    int i;
    if (argc == 1) {
        printf("Usage: %s <string to hash> [<string to hash>...]\n", argv[0]);
        return 1;
    }
    for(i = 1; i < argc; i++) {
        char * hashval = hash(argv[i]);
        printf("Hash of [%s] = [%s]\n", argv[i], (hashval?hashval:"Error"));
        if (hashval) {
            free(hashval);
        }
    }
    return 0;
}

(Return to top)