Openelec vs Raspbmc vs Xbian

Recensions

Update

In less than 12 hours after being public this article hit 1500 unique visitors. I received a lot of comments and thus an update is needed. You will find original part that has been corrected under stricke through text.

It’s now been a month that I’m playing with a borrowed (thanks to my flatmate Spencer Matteson) Raspberry Pi. Didn’t have time at the begininnig but as soon as I started connecting to a tv and sharing with all my flatmates it was fun from the first second.

I’ve been reached a stable configuration after X ( read as x big) number of researchs, hacks and tries. In this blog post I want to share my impressions and my researches about Raspberry Pi as a mediacenter with anyone that reads this blog post.

Raspberry Pi

It’s a cool toy, nothing to say about that. Even if people on the internet says that it lags in playing 1080p video in all my tests I didn’t find any lag. They pop up as soon as you start playing a 1080p video and doing netwoking intense operation like starting a rsync with a remote server or downloading 4/5 torrents.

If you have 25$ buy it. It’s worth the time and money you will spent on it. Due to the fact that it gets powered by a micro-usb connector (we are talking about 1A at 5V) don’t expect to connect a portable usb disk without power supply to the Raspberry. It will simply not going to work well and keeps rebooting just because power is too low to keep on the hard drive.

In my configuration I just connected to the Raspberry pi a ethernet cable, a HDMI cable and a usb hard drive with its own case and power supply.

Distro challenge

Openelec

Version tested: 3.0 RC2

Openelec was the first distribution I’ve tried. It’s easy, smooth to install and to use. It just works (TM). It has only one goal: be the best mediacenter distribution. And It just does it very good. With openelec you don’t have to worry about updates but as a con, you miss a package manager and everything of a linux distribution actually. It’s very stable and if you just need a mediacenter I will use this.

There is openssh installed but you will barely need it, I just used for a couple of reboots. The XBMC repository is already set and you can install any kind of software that is available directly from the UI. The problem is that you will not find anything advanced like a webserver or any other package you will find in other distributions. The farest I went while using it was to install transmission bittorrent client directly from XBMC repositories from the UI. It went well and worked pretty smooth. But transmission just uses http and does not support https so exposing it outside your personal network is not wise. And since you don’t have a webserver you can’t actually put an apache configured as https trasparent proxy in front of transmission.

Also I was really disappointed when I figured out that It doesn’t support changing ssh root password as its FAQ states. Since I soon wanted to do more with Raspberry pi that a simple mediacenter this was clearly an issue. I than tried Raspbmc.

Raspbmc

Version tested: 1.0 final installed on 3th of Feb 2013

Seriously. Don’t do that. Don’t spend your time in trying to figure out to make this distro work. It’s bloated. As soon as I installed I liked the really nice self installer and the cool splash screen while the system is loading. But apart from that, underneath you have a broken apt and package repository.

I tried to install the *byobu* package. When it was installing it’s dependence, *screen* there was a detected /deadlock in the init system/ so it refuses to install it. What?! (as a comment demonstrated this issue seems to have been fixed after this post. I’ll try myself as soon as I’m nearby my Raspberry). After that I installed apache2 and transmission and linked the two together making apache2 being a transparent ssl proxy. It didn’t work. It appears that Raspbmc mounts the drives as the pi user which is the one that runs the xbmc application using writing permission only for the pi user. The problem is that transmission runs as debian-transmission user. So it is unable to write on my external usb disk! After tweaking with /etc/udisk-glue.conf configuration I managed to mount the devices as root and assigning a write permission to all the users.

(The best part was when I tried to install some software from XBMC repository. I couldn’t because the repo was empty. Bart Otten, webmaster of Xbian.org and Superrepo.org contacted me privately saying that this issue is related to the XBMC repo. It can happen in any distribution that has XBMC. So if you don’t find software in your XBMC addon catalogue, just wait until the repository is up again.)

It may be the fact that Raspbmc is a one person project but the fact is that I found it really unstable and not flexible. If I were Sam Narzako (the man behind Raspbmc) I would open the project to more people or, better, contribute to the Xbian project.

Sam Nazarko politely commented that Raspbmc is not a one man project. I apologize for the misunderstanding but I got this information from the raspbmc.com about page that states that “Raspbmc is created and maintained by Sam Nazarko, a 19 years old student from London”.

Xbian

Version tested: 1.0 Alpha 4

Xbian was really easy to install. After downloading its image and put in a sd card as its website states first time that it loaded on the Raspberry pi to stretch the partition in order to use all the space in the sd card. The system was ready in less than five minutes and I was able to install byobu, apache2 and transmission without problems. Also the usb hard drive is automounted allowing anyone to create or read any file (chmod 777).

I then installed the XBMC watchdog plugin in order to keep the library up to date with new additions. The system resulted to be really stable.

I found Xbian to be the most mature mediacenter distribution for Raspberry pi. It is easy to install and maintain but it preserve the flexibility of a debian based distribution. The only annoying thing is a menu that appears to the user anytime he/she logs in with ssh. I don’t think that this behaviour is wanted by an user that is connected through ssh because it tries to simply management. But who connects through ssh is not a newbie and thus does not need any simplification. Instead this menu alter the expected behaviour that the user imagine about connecting through ssh.

Configuring apache2 as transparent proxy for transmission

In order to simplify configuration, the configuration for apache2 as transparent proxy for transmission is the following:

<VirtualHost *:443>
 <proxy>
    Order deny,allow
    Allow from all
  </proxy>

  SSLEngine On
  SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
  SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

  ProxyPass        /  http://127.0.0.1:9091
  ProxyPassReverse /  http://127.0.0.1:9091

</VirtualHost>

I’ve my new shiny mediacenter, how do I control it from my sofa?

You can have a Raspberry pi with all the software you want, but you will never have a mediacenter until you can control laying on your sofa. For xbmc, which is the base software in all the distribution I just spoke about, the remote control software is Yatse or Xbmc remote.

I tried both and I prefer Yatse. It’s much more complete and the UI is much more effective than the one in Xbmc remote. Also for functionality yatse seems much more powerful even if I didn’t analyze each specific functionality between the two remotes. You can install Yatse directly from goole play as like Xbmc remote.

They both work accessing your Raspberry pi via the network. All of the distribution I’ve shown have remote control enabled by default although both Openelec and Raspbmc use port 80 while Xbian uses port 8080.

I highly suggest to make Xbmc scan through your media files in order to populate your library. In fact library is easy accessible, sortable and playable from yatse and It will save a loot of file in finding the right file expecially when it’s in a directory polluted by other media files.

Alex, from a comment, says that XBMC Remote for iOS works very well, I didn’t test it personally. YYou can find it on the app store.

How to run OpenVpn and a webserver (Apache, Nginx, Cherokee, etc) in the same port

HowTosysadmin

http://i2.wp.com/www.hotforsecurity.com/wp-content/uploads/2012/10/The-New-York-Times-Website-Blocked-in-China-1.jpg?w=645

If you have ever used OpenVpn you know that a lot of filtered network do not allow to connect to other ports rather than 80 and 443 TCP. If you have your OpenVpn server in the same machine alongside with a Webserver this may be a problem because you want to use port 80 and 443 for your webserver and also the openvpn.

OpenVpn developers are smart people and they knew that this situation is not so odd so they invented a special option to deal with it. Thanks the article at raniersblog.org I found the port-share option that just redirect all the packets that are not vpn ones to another host:port pair. This allows to run your webserver in a different port, let’s say 8443 and make OpenVpn run on the 443 TCP. If packets are not for the vpn they will get redirected to the 8443.

To do so just configure your webserver to listen to port 8443 and add this configuration to the openvpn configuration file (usually /etc/openvpn/server.conf):

proto tcp
port 443
port-share 127.0.0.1 8443

Then restart openvpn and your webserver and you should be all set.

Python Bunch object or How to make dictionaries act like objects

Programming

Python dictionaries are really powerful, but sometimes you need an object that is a dictionary but it’s also an object which keys are methods of the same object.
Now think if you can have a class called Bunch that makes something like this possible:

bunch = Bunch(a = 7, b = 3, c = 'hello')
In [5]: bunch.a
Out[5]: 7
In [6]: bunch.b
Out[6]: 3
In [7]: bunch.c
Out[7]: 'hello'

But it’s also more powerful than this:

In [9]: d = {'a.b.c':1, 'a.c':2}
In [10]: bunch = Bunch(**d)
In [11]: bunch.a
bunch.a      bunch.a.b.c  bunch.a.c
In [13]: print bunch.a.b.c
1
In [14]: print bunch.a.c
2
In [15]: print bunch['a.b.c']
1

You can also use the dot notation to describe an hierarchy of objects and the Bunch object will take care of it

It’s not a dream, I’ve just implemented this python Bunch Object and this is the code:

class C(object): pass

def rec_getattr(obj, attr):
    """
    Get object's attribute. May use dot notation.
    """
    if '.' not in attr:
        return getattr(obj, attr)
    else:
        L = attr.split('.')
        return rec_getattr(getattr(obj, L[0]), '.'.join(L[1:]))

def rec_setattr(obj, attr, value):
    """
    Set object's attribute. May use dot notation.
    """
    if '.' not in attr:
        setattr(obj, attr, value)
    else:
        L = attr.split('.')
        if not hasattr(obj, L[0]):
            setattr(obj, L[0], C())
        rec_setattr(getattr(obj, L[0]), '.'.join(L[1:]), value)

class Bunch(dict):
    def __init__(self,**kw):
        dict.__init__(self,kw)
        self.__dict__.update(kw)
        for k,v in kw.iteritems():
            rec_setattr(self, k, v)

Comments, suggestions, patches are welcome 🙂

download list of files from a remote ssh/sftp server

Programming

I’ve approached this problem before. I’m on a server and I want to copy some directories from another server using scp. You have to write down all the paths of the directory you want of the first server, then go to the second, then do a command like:

scp -r server1:"directory1 directory2 directory3 ..." .

Which is easy with simple filenames but as soon as you need directories in different paths or long named directories it will be pain. And you don’t have shell completion. You could say: “Set up key authentication and you will have shell completion on the remote server”. Well, I don’t wanna set up keys for a server and I want a more practical way to solve this.
I wanted to connect through ssh from nautilus (just type ssh://server on the path bar), surf my remote directories, then download what i need to a second server. Well I did a python program to do that.
It’s pretty straightforward. You connect with nautilus to a server, select all the directories you want then copy them. Open the shell of another server, then do: python3 download.py -s server1 CTRL+V
The CTRL+V will copy all the filename absolute paths straight to the shell. Just hit enter, it will ask for password, just give it and you are done. It will start downloading all the list of directories of files you want from server1
This is the python program that makes this possible.

<pre>
#!/usr/bin/python
# -*- coding: UTF-8 -*-
'''
This programs allows easy copies of remote directories to the local path using scp

        usage:
                python3 download.py [options] list-of-files

        options:
                -s, --server server:set the server instead of deducing it

@author: Vincenzo Ampolo <[email protected]>
'''

import sys
import os
import getopt

import urllib.parse

def shellquote(s):
        """escapes the given string s such that it can be used safely in a shell
        """
        return "'" + s.replace("'", "'\''") + "'"

def main(argv=None):
        if argv is None:
           argv = sys.argv

        opts, extraparams = getopt.gnu_getopt(argv[1:], "hvs:", ["help", "server"])

        verbose = False
        server = None

        for o, a in opts:
                if o == "-v":
                        verbose = True
                elif o in ("-h", "--help"):
                        print(__doc__)
                        sys.exit(0)
                elif o in ("-s", '--server'):
                        server = a
                else:
                        assert False, "UnhandledOption"

        l = []
        for path in extraparams:
                url = urllib.parse.urlparse(path)
                if not server:
                        server = url.host
                quoted_path = shellquote(urllib.parse.unquote(url.path))
                l.append(quoted_path)
        files_to_download = ' '.join(l)
        cmd = 'scp -r %s:"%s" .'%(server, files_to_download)
        os.system(cmd)
        return 0

if __name__ == '__main__':
    sys.exit(main())
</pre>

This ban is going to hurt you so badly Apple

Random Thoughts

Good artists copy; great artists steal. (cit. Steve Jobs, Apple founder)

Back in the 70′ through the 00′ Apple was a very good company. It was surely one of the best technology innovator in the world. Remember the Apple I, the all-wood computer that made the personal computer being real? It was amazing. I never saw one for real but It’s in the hall of fame of the computer history.

Do you remember the Apple II, one of the best Apple products, the one that was able to sell $79m in the old 1979? That was innovation. In a world dominated by large scale computer they found a new market: personal computer. Do you remember 1984? There was this commercial:

[youtube http://youtu.be/OYecfV3ubP8]

I just saw that and I see a company that would like to make genuine innovation, compete with products not with lawyers. Another company was very good at it, and it’s called Microsoft.

Yesterday I read that Apple wins sales ban on Google galaxy nexsus. It seems that after Steve’s administration Apple turned to be much more like Microsoft. They stopped competing with products and they choose to compete with lawyers. It seems that the fuel that made Apple be one of the best tech companies of the world is gone with its beloved founder Steve.

I think that was a very bad move. It seems like Apple is maybe going to be an empty box without any sort of innovation trying to make money out of awful patents acquired during time. I’m so sorry for all the Apple fanboys, these moves are hard to be defended in terms of technology innovation.

I’m just curious, was Apple really thinking that it could defeat android with this move?

How to install pymatlab

HowToubuntu

For a university project i had to deal with a web application written in python and some computations that are matlab based. Of course my webapp was in python so I had to find a library to interact with python. I’ve found pymatlab to be a good library for this goal.

Installing pymatlab was a bit tricky. I had to install matlab on Linux, then install the python-numpy package and then install pymatlab with this sets of commands


sudo -s

export LIBRARY_PATH=/usr/local/MATLAB/R2011a/bin/glnxa64/

export PATH=$PATH:/usr/local/MATLAB/R2011a/bin/
 export C_INCLUDE_PATH=/usr/local/MATLAB/R2011a/extern/include/
 apt-get install csh
 pip install pymatlab

Then I was able to run matlab inside my python programs.

Why IronPython is shit…

Coding

I’m not a C# fan, I’m more a python fan though. Anyway. At work I had to make C# be able to call python functions. So I started googling. I found some nice posts from:

Run python from C# and vice versa

But thanks to .Net 4.0 It seems that a lot of troubles have been resolved introducing the new dynamic type in C#. With that type I can finally do some work in python then export any class to my C# world.

There are still some issues in using any python module: Ironpython is not compatible with the normal python interpreter, or CPython. You can use Ironclad to make Ironpython run C CPython modules but they don’t support Python 2.7 yet and the last update from their release is from  07/2010.

Install IronPython

After installing IronPath you have to modify windows PATH system variable such that you can execute ipy in a shell and get a IronPython console. To do so you have to add the following two lines at the end of your PATH variable on Windows:


;c:Program Files (x86)IronPython 2.7;c:Program Files (x86)IronPython 2.7Sciprts

Install Setuptools

Tried to install distribute. Of course it didn’t work. It crashed. I’ve successfully managed to setup setuptools but their support is broken and i’m unable to install simple python eggs due to multiple error.   These are the steps that I’ve followed

c:UsersvinzDesktop>ipy distribute_setup.py
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.2
7.tar.gz
Extracting in c:usersvinzappdatalocaltemptmpanru2x
Now working in c:usersvinzappdatalocaltemptmpanru2xdistribute-0.6.27
Installing Distribute

Process is terminated due to StackOverflowException.
Something went wrong during the installation.
See the error message above.

Use setuptools instead http://peak.telecommunity.com/dist/ez_setup.py

Open a priviledged shell http://www.howtogeek.com/howto/windows-vista/run-a-command-as-administrator-from-the-windows-vista-run-box/


ipy ez_setup.py

Since there is no support to zlib in ironpython you cannot install any egg anyway even if you installed setuptools.

Install an egg

I’ve managed to install setuptools, however installing a simple package fails because .pth files are not managed correctly. So the best way of installing eggs is just to take their source, maybe unpacking an egg since it’s just a zip file, and copy directly on the IronPython site-packages directory. I’ll still describe the process I did just in case some reader sees that I’ve done a mistake.

To install an egg, just unzip it and run:


ipy -X:Frames setup.py install --user

The –user will install the package for user only. To install the package system-wide you have to install from an administrator shell.

The best thing to do to install an egg is just to unpack it, it’s a zip file, then copy the directory of your library in C:Program Files (x86)IronPython 2.7Libsite-packages . That directory will be included in sys.path of every ipy interpreter.

First problems

Just run a first example which uses ElementTree and i got:


NotImplementedError: iterparse is not supported on IronPython. (CP #31923)

Surfing a bit the web I found that It’s a bug from Dec 2011 http://ironpython.codeplex.com/workitem/31579

So I’ve downloaded and installed elementTree, It will install on normal Python’s site-packages, so you have to move the folder unver IronPython’s site-packages. After doing so, when you need to import etree just do:


import platform
if platform.python_implementation() == 'IronPython':
from elementtree import ElementTree
else:
import xml.etree.ElementTree

This code will import the just installed elementtree library or the system library one depending if you are on normal Python interpreter (CPython) or IronPython.

Conclusions

From this experience I’ve learnt many things:

  • IronPython is not a full python interpreter: it lacks many basic support like the one for setuptool and distribute
  • IronPython is very bugged, even the standard library is bugger. Thus import as external module everything you need.
  • IronPython cannot use the vast majority of python modules because to support .NET jit it breaks CPython compatibility

Thus. Keep away from it! It’s just crap. Microsoft’s dream of .Net simply doesn’t work for python. Yeah, you can access Python functions and class from C# but you miss the goal that makes people use python: a very huge amount of libraries and packages that are easy to install and integrate. Sorry Microsoft, it’s yet another #Fail

How to quickly share a git repository

HowTo

Today I had to simply share a local repository with a collegue. It was pretty easy, just do:


cd MyGitProject
git daemon --reuseaddr --verbose  --base-path=. --export-all ./.git

the reuseaddr option will reuse connection, verbose will let you see what will happen, base-path is for using relative paths and export-all is to export all the repos in the .git folder. Easy, isn’t it?
Tell your friend/collegue to do

git clone git://your-ip-address-here/ MyGitProject

And it will start downloading the new repo 🙂 Amazing, isn’t it ? PS: do not use bad GUIs, they will mess the command line to get the repo. just open a shell, even on windows, and type that command.

How to put aptana launcher on the ubuntu dock

HowTo

Today after using vim for a while for my web development, I decided to give a try to Aptana. So I just downloaded it and… I liked it. I use the Super+N feature of Unity a lot, I think it’s a killer feature of this DE. Well. I wanted to put aptana in the dock too.

Doing so is prety easy. Just unpack the aptana package you just downloaded from aptana.com in whatever folder you like. I usually put this kind of custom apps in ~/Apps . now enter into the newly created aptana folder and create a file named aptana.desktop.

Put this content into that file:


[Desktop Entry]
Type=Application
StartupNotify=True
Name=Aptana
Comment=Aptana Studio 3
Icon=/home/goshawk/Apps/Aptana_Studio_3/icon.xpm <- replace this whit the ABSOLUTE path in which you unpacked aptana
Exec=/home/goshawk/Apps/Aptana_Studio_3/AptanaStudio3 <- the same applies here, replace with the ABSOLUTE path
Terminal=false
Categories=Development;IDE;

Give that file executable permissions. A chmod +x aptana.desktop should be fine. You can also right click on the file from nautilus, go to properties and then flag execute. The icon of the file will change and will be the aptana icon. Your launcher is now ready. Just drag and drop it to the dock and right click on it and select lock to launcher. Well done.

If you want it to be also accessible from the unity text launcher (the one that appears when you press the windows logo on your keyboard) just copy the aptana.desktop file to ~/local/share/applications. Your aptana will be also available from the text launcher now.

Set up a full redundant btrfs storage under ubuntu linux

ubuntu

Hard disks are going to be cheaper and cheaper and we are going to need even more space. So why not build your own cheap storage? Thanks to linux 3.1 and 3.2 and its btrfs support is now very easy to setup a fully redundant, scalable, storage made of many hard drives. Set up a btrfs storage is really easy. I’m using btrfs because it performs really good in a lot of scenarios like oracle demonstrated. Do not rely on old benchmarks, Btrfs has been out from a while now, and early versions are not comparable with old ones. One of the best features of this filesystem is scalability: you can start with one disk and then attach new disks as soon you need them. You just attach and add to the btrfs volume the new hard disk. That’s it.

To create a fully redundant hard disk just do:


goshawk@pluto:~$ sudo apt-get install btrfs-tools

goshawk@pluto:~$ sudo mkfs.btrfs -d raid1 /dev/sdb1 /dev/sdc1

With:


goshawk@pluto:~$ sudo btrfs filesystem show

Label: none uuid: 627e8d8a-8808-46fa-a237-875cd621f7f3
 Total devices 2 FS bytes used 28.00KB
 devid 1 size 931.51GB used 2.03GB path /dev/sdb1
 devid 2 size 931.51GB used 2.01GB path /dev/sdc1

Btrfs Btrfs v0.19

You can see the newly created filesystem. You will see also the UUID. That’s an important information since we can setup automount at startup thanks to that info.

Suppose that you want to attach the newly created storage to /storage and you want it to be mounted by default as soon as the computer starts. You just do add this line to /etc/fstab:


UUID=627e8d8a-8808-46fa-a237-875cd621f7f3 /storage btrfs defaults 0 0

Happy storage!


 					

						
			









						
							

								
							


											

					

															

				

			

		
 
	
 

 
 

	

	


	
			


























		
		
%d bloggers like this: