Android + Mediatomb + UPnPlay = Music Anywhere
Posted by Jack on 2011-06-29 at 23:31Tagged: software
To be clear, this post is about hitting a UPnP (streaming media) server securely from outside of the LAN. If you just want LAN access, all you need is MediaTomb and UPnPlay. No special config.
In addition, as some Redditors pointed out, you can use Subsonic for an easier open source solution. I already had MediaTomb and I wanted to use it. In addition, this solution should work with all other UPnP devices (which Subsonic / Google Music don't run on) and for video streaming (with vPlayer, or another app UPnPlay can talk to) as well.
NO ROOT REQUIRED
- Introduction
- MediaTomb
- SSH
- External Connection Setup
- UPnPlay
- Secure Connection via SSH with ConnectBot
- Finally, UPnPlay WAN
- Concluding Notes
Introduction
So Juliette and I ditched our respective cellphones (hers the aging dumbphone LG Lotus, mine the hard fighting but obsolete HTC Hero) with EVO3Ds, or 3VOs as the internet has nicknamed them. The phones are really cool, but I also noted that HTC advertised them as DLNA capable. This support isn't required for these steps, they should work anywhere that UPnPlay works (Android 1.6+ AFAICT), but it was the inspiration for the idea.
DLNA is a collection of companies agreeing on standards, and UPnP (Universal Plug 'n Play) is their set of standards for device cooperation. A lot of devices have come out in the past couple of years that interoperate. Very cool, although my first tangle with UPnP was with my PS3 a couple of years ago and it was very frustrating because the PS3 is a mis-mash of bad formats and pointless restrictions. Fortunately, though, that meant that when I started to experiment with the UPnP capabilities of the 3VO, I had a good starting knowledge. It still took me a bit of time to figure out how to be as remote as I wanted to be though. MediaTomb and the HTC Sense music player work pretty well together over WiFi but the Sense player has some drawbacks.
- Can't play Oggs. This was a major WTF. I couldn't figure out why it wasn't listing my music. One of the reasons I bought Android (among many) was that it had better support for open formats, particularly Ogg Vorbis, which is shown to have better quality at lower bitrates (very important when streaming) and, even better, it's a totally free and unencumbered format. For an interim, in the house solution I settled on MP3 which the Sense player picked up immediately.
- No built-in WAN support. This is typical of a lot of players, and the HTC player (along with a lot of them in the Market) won't even let you try to hit a remote server unless you're on a WiFi connection. That's all well and good, but sucks if you're planning on streaming while driving or any other place outside of your home.
Given those disappointments, I tried a lot of approaches to solve the problem. One approach that was suggested in various forums was to setup a VPN, which would allow the Sense player to think it's on your home network and make things like UPnP server discovery work out of the box. That's a huge pain in the ass to setup, especially if you're security minded, and most importantly it didn't work. I'm not a networking expert so it's possible that my VPN setup was screwed up (although my Android was able to connect), but it was hard enough in the end that I didn't want to maintain the solution anyway. Another solution I thought of would just be to run an NFS server, or another remote filesystem but those require root, which 3VOs don't currently have despite a promise from HTC to unlock soon.
So, without further ado, the rootless way to stream your media anywhere securely.
NOTE: Non-Linux users can get a bit of strategy from this post, but not very many details. There are Windows UPnP servers that will work insecurely in this manner, just not MediaTomb, and there are some Windows SSH servers, but I don't know anything about them. Unless you think you're confident enough to translate the instructions, turn back now.
MediaTomb
The first thing you have to do is setup a server to serve your content UPnP style. My favorite piece of software for this is MediaTomb. It's free, it's open source, and I already had familiarity with it getting the aforementioned PS3 to work with it. It also has a lot of neat transcoding options, which were invaluable for the PS3, but fortunately aren't necessary for streaming Oggs and MP3s this way. If you're serving FLACs (which are pretty high bitrate for streaming over 3G/4G) or another exotic format that your phone won't play then I refer you to the MediaTomb wiki article on transcoding and suggest that you judiciously use Google.
Installation
Most modern Linux distros have a mediatomb package. Throughout this post I'll assume Arch Linux, but the translation between pacman and apt-get or yum should be pretty straightforward. From the terminal:
$ sudo pacman -S mediatomb
Configuration
I chose to run MediaTomb as my user because it's better practice than having root daemons opening ports, even if they're secure. Just running mediatomb from the command line will make it generate the default user config (~/.mediatomb/config.xml)
along with an empty database file. The output of the daemon is quite useful and the default config is basically going to be right for our purposes. There are just two suggested tweaks to modify the default config:
Port and Interface
You're going to need to specify a port, because Mediatomb will use one randomly if you don't. That's usually not a problem because generally these servers are "discovered" by the streaming device, but in our case (usage from WAN) we have to do the discovery step for the application and you don't want the port to change. In addition, you want to be serving this content to your external network device, in my case (and most cases for wired connections) eth0
. I added the following to my config (~/.mediatomb/config.xml)
:
<?xml ...>
<config ...>
<server>**<port>49153</port>
<interface>eth0</interface>**
...
That's pretty much it for the config.xml
. If you're shy of opening an editor or messing with XML, both of these settings can be changed with command line switches when you start the server as well. You'd just use the -e <interface>
and -p <port>
options on invoking in the next step. Being familiar with the XML config is only really necessary for more advanced options, like transcoding.
Starting the server
Starting the server is as easy as invoking MediaTomb from a terminal:
$ mediatomb
It will output some useful information. In particular you want to see:
INFO: Configuration check succeeded.
INFO: Initialized port: 49153
INFO: Server bound to: [interface IP]
INFO: MediaTomb Web UI can be reached by following this link:
INFO: http://[interface IP]:49153/
Where interface IP is most likely your LAN IP. Mine is 192.168.1.25
for example.
Adding media
Now for the most important (and fortunately easiest) part: getting MediaTomb to serve up some content. MediaTomb has a nice and simple web interface that runs on the same port as the server. As it says in the above output you can hit the interface with the link http://[interface IP]:49153
. Just paste that into the address bar of your favorite (Javascript capable, sorry lynx) browser and you should be greeted with a screen like this:
I won't break down the details very much (this post is only tangentially about MediaTomb), but you initially go to the Filesystem section (the one I'm on in the screenshot), browse to your media using the lefthand tree, select the root directory you want to serve up (in my case music-ogg
) and then you hit the add button (looks like a plus in the upper right corner), and optionally tell MediaTomb to watch it (show autosync options with the plus with the circle of arrows around it). After setting that up, you can see the files that are being served in the Database section. The Database section is occasionally useful as it can give you details about how MediaTomb is using a file. In particular it's MIME-type which is how UPnP clients figure out which files they can play.
I setup mine such that it recursively monitors my music directory with inotify, which means I can drop files into the directory and they'll automatically be served. This is probably the best way, unless you want to be more selective about what you're serving up.
It may take a bit to add your whole directory to the database, but the server is actually operational as it adds the directory, so don't let that deter you from continuing.
SSH
The next step is all about security when you connect to MediaTomb. You should install an OpenSSH server. Most likely your install already has it, but it might not be running. If it doesn't have it, then you should install it and start it:
$ sudo pacman -S openssh
$ sudo /etc/rc.d/sshd start
If you're running pretty much any distro other than Arch, you'd install openssh
or openssh-server
and use another /etc/init.d/
script instead. Look for sshd
or openssh-server
. Check your distro's wiki or Google if you need help starting the SSH daemon.
By default, the SSH server runs on port 22.
We'll connect to MediaTomb via SSH later in the post.
KeepAlive
There is one foible with SSH that should be ironed out now too. We're going to setup a long term connection to the server over SSH, but a lot of routers and hardware will terminate a connection if it's idle for too long. That's not a problem if you're going to be streaming over the connection the whole time, but what if you stop and start again? The answer is KeepAlive which, as the name implies, keeps the connection alive by sending empty traffic every so often to inform hardware that the connection is still necessary.
UPDATE: I believe I got this wrong on the initial posting. In order to enforce this from the server side, instead of the client side (which is what ~/.ssh/config
affects), you need to add the following to /etc/ssh/sshd\_config
ClientAliveInterval 60
ClientAliveCountMax 3
This will send the client some traffic every minute to keep the connection alive, and will timeout the connection on the server side after 3 packets have been lost, for a total of 3 minutes of the client not responding (i.e. not returning the packets which is different than just sitting idle).
External Connection Setup
If you're already familiar with opening ports on your router and addressing your LAN from outside the house, just forward your SSH port to the right machine and skip this part.
The first is that you're going to need a way to address your home computers. I use DynDNS for a free dynamic hostname, and ddclient to keep the dynamic hostname up to date automatically.
You also need to setup your router to automatically forward traffic it receives on the SSH port (22 unless you changed it) to your server. Look for options in your router config like "Applications and Gaming" (which is a stupid name for it, but that was it on my Linksys) or, more simply, "Port Forwarding". Setup TCP forwarding to your machine running MediaTomb and SSH. This is made much easier if you either have your computer setup for a static LAN IP, or configure the router to hand out a consistent IP with DHCP to the server's MAC Address so that the port forwards are always to the right machine.
This is pretty standard fare for home servers, but if this isn't enough to figure out what you need to do, DynDNS has a little walkthrough that should help get you setup. They point you to portforward.com which will try to sell you some BS tool to do it automagically, but once you skip their advertisements they actually have decent information on a lot of router models. For all of these steps the only thing you should need is a browser, so don't despair at the IE screenshots =).
UPnPlay
I searched for a long time for an app that supported WAN connections to MediaTomb and UPnPlay is the only option that I've found. It's interface is a bit clunky, but it's functional. You can grab it by searching for UPnPlay in the market or using this QR code via Appbrain:
Test MediaTomb over LAN
This is optional, but getting UPnPlay to detect over LAN will make the WAN configuration much easier
The first thing you should do when you get UPnPlay installed is test that your MediaTomb setup works via LAN. Get on to your WiFi, fire up UPnPlay and it should automatically detect MediaTomb. Under it, you should be able to access your media in various forms, either by file path or by a number of pre-defined groups (like Audio > Album, etc.). If UPnPlay doesn't recognize MediaTomb, you're in trouble, I would suggest checking the mediatomb
output, or running mediatomb --debug
and seeing if the server even recognizes that it's being connected to. If not, then your first step is to figure that out.
If you have to debug this step, I suggest quitting (hit back on the library list and it will confirm quit) and restarting UPnPlay each time you test a change. Supposedly you can refresh the devices from the menu, but I've had no luck with it.
Directories, but no media
If you can browse your directories, but none of the directories have any files, then UPnPlay most likely isn't recognizing the media format. As I mentioned above, MediaTomb has a lot of nice transcoding options that I won't cover here, but they can be used to present any sort of weird file as one that your phone can play, and transcode that file on the fly when requested. Just to test everything else is working I'd suggest getting some vanilla MP3s/Oggs and adding them to MediaTomb. Those should show up without any special settings.
Setting up a WAN server
If everything above was hunky dory and you can access all of your media over WiFi, excellent. At this point, we're going to prepare a WAN server definition for UPnPlay that won't be used immediately but will make your life easier in a couple of steps.
First, enable WAN server:
Menu > Settings > Media Sources > WAN Server
Then, while UPnPlay has your MediaTomb server detected, long press the MediaTomb item in the library list and select "Clone WAN Server" You needn't make any changes except changing the first box after "Host" to "localhost" or "127.0.0.1" instead of the IP that's there. Hit OK and we'll save that for later. Go ahead and quit UPnPlay too (hit back on the main screen, it will prompt you to quit).
Secure Connection via SSH with ConnectBot
At this point, you should have a functional MediaTomb installation and UPnPlay working (over WiFi for now). Now we're going to turn off WiFi and use Android's excellent SSH client, ConnectBot to securely connect to the server. First things first, install ConnectBot by searching for it on the Market or using the following QR code:
Using ConnectBot you should be able to access your server, with WiFi off using the dynamic hostname or IP you setup above. From the initial screen, 'ssh' should be selected in the combobox and you specify user@external-hostname
for the connection. When you input that it should attempt to connect, prompt you for your user's password, and drop you in a shell. You don't need to do anything with the shell.
If you're unable to connect, most likely your hostname/IP is wrong (or not setup), or the port forward from your router to your server is messed up.
Setup a Port Forward
Here's the cool part. We're going to forward the server's MediaTomb traffic (port 49153 in my example) directly to your phone. The traffic will be encrypted well, and obviously you have to authenticate to get it setup so it's much more secure than just letting MediaTomb accept anybody's connection. Not to mention that leaving MediaTomb unencrypted and accessible to all is most likely illegal, depending on your country's view of file sharing.
To do this, while you're looking at your home server's shell, hit:
Menu > Port Forwards
And then, to add one:
Menu > Add Port Forward
The nickname doesn't matter. The type of port forward is "Local" (i.e. on the phone), and the source port should be 49153. The destination is a bit trickier. It needs to be how the server would refer to the port if you were logged into it. Basically you want to set it to "[Server's LAN IP]:49153", where that IP should correspond with the IP in the MediaTomb output when you invoked it. In my case the value is "192.168.1.25:49153". If you've got your /etc/hosts setup correctly it could also be "hostname:49153".
Once you have the port forward setup, from your phone's browser you should be able to access the MediaTomb media screen without WiFi by going to http://localhost:49153
. That translates to "this device (localhost, the phone) port 49153" and will work if the forward is active. NOTE: Make sure you include the "http://"! If you just put "localhost:49153" the browser will interpret this as a Google search!
Finally, UPnPlay WAN
To double check, at this point you should be able to:
- Use UPnPlay over the LAN
- Setup an SSH port forward to your server, with no WiFi
- See the MediaTomb web interface by going to "http://localhost:49153" (with the http part!)
If you can do all of those things, you've basically got it! Now you just need to fire up UPnPlay when you're not on the WiFi. In the library list you won't see your MediaTomb server. If you did the clone step above, you'll see "MediaTomb (WAN)". Hit that, it should connect, and cause a "MediaTomb" item to appear. Then you just access your music through that normal "MediaTomb" item. Golden!
If you didn't / couldn't clone
If you're trying to setup the WAN server option, but for some reason you're away from the LAN and couldn't do the clone step above, then you can manually add the WAN server.
First enable the WAN Server (this was covered in the clone step too):
Menu > Settings > Media Source > WAN Server
And then, from the library list (the initial screen):
Menu > WAN Server
Then fill out the fields:
**Name** Should be the same as the LAN server (MediaTomb by default)
**Host** localhost : 49153
**Description** /description.xml
**UDN** uuid:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
Description is a path to the XML description of the server, so that's the same for all MediaTomb servers. The UDN is the real pain in the ass and why cloning is so much more convenient. You can find the value for your server in the MediaTomb config.xml
where it's everything between the UDB tags including the uuid: part.
UPnPlay Troubleshooting
UPnPlay is a bit finicky. As I mentioned before, the interface is clunky but functional. Here are some tips:
- UPnPlay seems to be unaware of network changes. If your connection changes (i.e. WiFi to 3G, 4G to 3G etc.), quit UPnPlay (hit back until it prompts to quit) and restart. Device refresh seems a bit odd and I spent a bit of time trying to connect over 3G only to realize that when I hit "MediaTomb" is was still trying to hit the LAN version. Tangentially, having the WAN server variant just spawn something that looks just just the LAN variant which you actually use is very confusing.
- If you can see your media files remotely you're all set. Starting a song may take a bit to actually get traction (i.e. it fails to play the first five seconds a couple of times), but be patient. After the first song plays though I usually don't have trouble moving on to the next track. Also, I haven't tested the "Near Gapless Playback" option, but this might help between tracks.
- If you do lose connection without switching networks, check to make sure that your ConnectBot session is still active. If it's closed, the port forward obviously won't work.
Concluding Notes
I didn't cover automatically starting these up on boot. Currently, I start SSH with Arch's /etc/rc.conf
. Then, from /etc/rc.local
I use sudo jack -c mediatomb
to run the MediaTomb server as my user. rc.local
is universal, as far as I know, but for starting daemons check your distro's wiki.
I've tested this a bit around Austin. I haven't had much luck (in general on Sprint) with 4G, but 3G seems to work consistently and stream Ogg files without taking too much of my upload bandwidth or stuttering. I've also tested it connecting from foreign WiFi and UPnPlay seems smart enough to use the WAN definition even if you're hooked to WiFi (thankfully).
Clearly this is going to be highly contingent on your bandwidth on the server. If you're in a low bandwidth situation, you can try transcoding into a lower quality stream but you might be better off with buying a bigger SD card =).
I haven't attempted to get cover art to work. I also haven't attempted videos, although UPnP (and UPnPlayer with some third-party video apps) supports it. I can't see a reason it wouldn't work however. I can say that streaming remotely, the Amazon cover art lookup doesn't seem to work. I don't have a problem with this as I'm more concerned with bandwidth and performance, but don't expect it to work out of the box.
Lastly, Bebopfreak, UPnPlay's developer, will not take support requests on this feature. I, on the other hand, would be happy to attempt to give pointers in the comments and update this little walkthrough with any finer points.
Good luck!