Zero to Play with Linux Neo4j, Play2 and nginx

This is a guide for taking your (debian like) linux system from a fresh install to staging server.

sudo -s
apt-get update
apt-get dist-upgrade

# install Sun java 7
echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list
echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys EEA14886
apt-get update
apt-get install oracle-java7-installer

# install scala
apt-get install scala

# build tools
apt-get install build-essential

# install recent nodejs
sudo apt-get update
sudo apt-get install -y python-software-properties python g++ make
sudo add-apt-repository ppa:richarvey/nodejs #or sudo apt-get-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs nodejs-dev npm

# install nginx
apt-get install nginx


# get play
wget http://downloads.typesafe.com/play/2.2.1/play-2.2.1.zip

# unzip this somewhere
vim .bashrc

# add play to path
export PATH=$PATH:/path/to/play

# install neo4j

# start root shell
sudo -s
# Import our signing key
wget -O - http://debian.neo4j.org/neotechnology.gpg.key | apt-key add - 
# Create an Apt sources.list file
echo 'deb http://debian.neo4j.org/repo testing/' > /etc/apt/sources.list.d/neo4j.list
# Find out about the files in our repository
apt-get update
# Install Neo4j, community edition
apt-get install neo4j
# start neo4j server, available at http://localhost:7474 of the target machine
/var/lib/neo4j/bin/neo4j start


# install git
apt-get install git

# clone projects
...
...

# make sure things are good
cd projects/dir/
play compile

# test running the server
play run

# compile staging package
play clean compile stage


# setup your init scripts to keep it alive
vim /etc/init.d/play.myapp

###################################################
## Init Script
###################################################
#!/bin/bash

APPLICATION_PATH=/home/user/projects/myplayapp

start() {
    echo -n "Starting"
    sudo start-stop-daemon --start --background --pidfile ${APPLICATION_PATH}/RUNNING_PID -d ${APPLICATION_PATH} --exec target/start -- -Dhttp.port=9000
    RETVAL=$?

    if [ $RETVAL -eq 0 ]; then
        echo " - Success"
    else
        echo " - Failure"
    fi
    echo
}
stop() {
    echo -n "Stopping"
    sudo start-stop-daemon --stop --pidfile ${APPLICATION_PATH}/RUNNING_PID

    RETVAL=$?

    if [ $RETVAL -eq 0 ]; then
        echo " - Success"
    else
        echo " - Failure"
    fi
    echo
}

  case "$1" in
    start)
      start
  ;;
    stop)
      stop
  ;;
    restart)
      stop
      start
  ;;
    *)
      echo "Usage: play-server {start|stop|restart}"
      exit 1
  ;;
esac
exit $RETVAL



## END INIT



#make it executable 
chmod +x /etc/init.g/play.myapp

# test it
/etc/init.d/play.myapp start

# now make it start on boot
update-rc.d play.myapp defaults


# finally lets get some proxy action
# edit nginx.conf adding servers for each of your projects...
vim /etc/nginx/nginx.conf


######### SAMPLE SECTION TO ADD in http{ ...
# add 1 sever per domain. Also note that you can have wildcards *
 server {
        listen       80;
        server_name  *.mysite.com;
               
        location / {
           proxy_pass http://localhost:9000/;
           proxy_set_header Host $http_host;
        }
# 404 ect...
}

# now restart nginx
/etc/init.d/nginx restart

Lets assume you had a number of sites on one server. You could have a number of play applications running on various ports. Then in nginx.conf you would define a sever rule for each site. Also MAKE sure you pass through the Host header or you will run into issues. (ex: secure_socaiul sending redirect urls as "localhost:9000"

Nginx as a proxy passthrough for BOSH

In a past post I talked about using nodejs and http-proxy as a way of routing your request to internal handlers. One thing that we want to do when working with XMPP from the web is to proxy request to a BOSH server running on another port. There are issues that arise here from Cross Origin Scripting or COORS. Basically if (stophe.js) does not make the BOSH request from the same domain and port, it will fail.

To get around this I added another rule into my nodejs routes. The rule looked something like the following.

var http = require('http'),
    httpProxy = require('http-proxy');
//
// Create your proxy server
//

var options = {
  router: {
    'domain.com/http-bind/': 'localhost:7070/http-bind/',
    'domain.com' : 'localhost:9000',    
  }
};

httpProxy.createServer(options).listen(80);

This did not work. I figure it has something to do with not passing through the correct host header.. thus still running into a CORS issue.

Solution Nginx

Download and install nginx What was required was to setup nginx to allow proxy-passthrough to BOSH. You need to edit your nginx.conf file and include something similar to the following.

server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;
  location /http-bind/ {
   proxy_pass http://localhost:7070/http-bind/;
  }

        location / {
    proxy_pass http://localhost:9000/;
    proxy_set_header Host $http_host;
  }
}

In this case we are taking traffic from port 80 and either mapping it to our Play Server (port 9000) or to the openfire BOSH connection (http://localhost:7070/http-bind/). This worked :)

Neo4JMembershipProvider

asp.net Neo4J Membership Provider

Dependencies

Neo4J Graph Database >= v 2.0.0 (Make sure it has label support)
Neo4JClient https://www.nuget.org/packages/Neo4jClient. NOTE: this package is auto installed as a dependency when you use NuGet

Install

NuGet package https://www.nuget.org/packages/Neo4JMembershipProvider/
This is a step by step install for a new MVC application.
First thing to do is to make sure that you have the latest NuGet installed in your Visual Studio. Again, make sure it is the latest version.

At this point make sure that you have Neo4J up and running and that you can connect to it. Assuming you have Neo4J installed on your localhost you should be able to connect via your web browser with the address.

http://localhost:7474

If you have everything up and running you should see a screen similar to this one.
Create a new MVC 3/4 web application. Once you are looking at the project files, right click on your refrences and choose "Manage NuGet Packages.."
This will open the NuGet Modal. Select "Online" from your list of sources on the left hand side. Next you can search for "Neo4JMembershipProvider" in the top right search menu.
Once you see the package in the main list. Click the install button.
Verify that the package installed by looking for a green checkmark next to the package. See image as refrence
Now we have to make a few changes to the configuration of our application.
First off we will need to modify our web.config file to include the following lines

<configuration>

  ...

 <connectionStrings>
    <add name="DefaultConnection" connectionString="http://localhost:7474/db/data" providerName="Nextwave.Neo4J.Connector.Neo4JClient" />
  </connectionStrings>
  <appSettings>

    ...

    <add key="enableSimpleMembership" value="false" />
    <add key="autoFormsAuthentication" value="false" />
  </appSettings>
  <system.web>

    ...

    <roleManager enabled="true" />
    <machineKey validationKey="C50B3C89CB21F4F1422FE158A5B42D0E8DB8CB6CDA1742572A48722401E3400267682B202B746511891C1BAF47F8D25C07F6C39A104696DB51F17C529AD3CABE" decryptionKey="8A9BE8FD22AF6979E7D20198CFEA50DD3D3799C77AF2B722" validation="SHA1" />
    <membership defaultProvider="Neo4JMembershipProvider" userIsOnlineTimeWindow="15">
      <providers>
        <clear />
        <add name="Neo4JMembershipProvider" type="Nextwave.Neo4J.Membership.Neo4JMembershipProvider" connectionStringName="DefaultConnection" applicationName="Nextwave" enablePasswordRetrieval="true" enablePasswordReset="true" requiresQuestionAndAnswer="false" requiresUniqueEmail="true" passwordFormat="Hashed" />
      </providers>
    </membership>

    ...

</configuration>

I have also included an image with the highlighted changes.
NOTE: that your connection string may need to change to point to your db location
The last thing you need to do is to modify your "InitializeSimpleMembershipAttribute.cs" This is located in your /filters/InitializeSimpleMembershipAttribute.cs
Make the following code changes

  [AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
  public sealed class InitializeSimpleMembershipAttribute : ActionFilterAttribute
  {
      private static SimpleMembershipInitializer _initializer;
      private static object _initializerLock = new object();
      private static bool _isInitialized;

      public override void OnActionExecuting(ActionExecutingContext filterContext)
      {
          // Ensure ASP.NET Simple Membership is initialized only once per app start
          LazyInitializer.EnsureInitialized(ref _initializer, ref _isInitialized, ref _initializerLock);
      }

      private class SimpleMembershipInitializer
      {
          public SimpleMembershipInitializer()
          {
              try
              {
                  WebSecurity.InitializeDatabaseConnection("DefaultConnection", "User", "Id", "UserName", autoCreateTables: false);
              }
              catch (Exception ex)
              {
                  throw new InvalidOperationException("Something is wrong", ex);
              }
          }
      }
  }

That should be all you need.
You will now be able to start your application and regester / login
Your users will be nodes in neo4J with the "User" label.
So for example you could list all your users with the following Cypher

MATCH u:User RETURN u;

Finally

My solution has only undergone a very small amount of testing.
However, if you have any trouble please contact me and I will be glad to help.
Thanks... and Enjoy the power of your new graph db :)

AngelHack Vancouver and Bluetooth Low Energy

AngelHack Vancouver and Bluetooth Low Energy

Event: details

This last weekend I participated at the AngelHack hackathon. What is this?  Teams got to pitch their idea and collect members that were interested in the same thing. You then had 24 hours to complete/hack your project and present it to the group.


The idea I wanted to work on included the use of a TI Sensor tag. This is similar to apples iBeacon technology and uses BlueTooth 4.0 Low Energy. The low energy means that the devices work for low range, but also that some will operate for over 2 years on a coin battery. We intended to use a number of TI Sensor tags and an android device to get accurate indoor positional data. Once we had an accurate position and an orientation for the device we planned on allowing you to interrogate your environment.  This environment interrogation would take form as a kind of Augmented Reality looking through the devices camera.  I had imagined the terminators red view of the world with constant printouts of scanned objects ;)

The plan for getting the BLE data was to fix the sensor tags to known coordinates (Lat, Lng) and then measure the field strength of the tag to the device.  Using a number of field strength values we could then interpolate to get an accurate position for the device.  THIS DID NOT WORK :(

As it turns out the TI Sensor tag is incapable of providing data that can be used in this manor.  About the only predictable use of the field strength was to determine if you were really closet to the device (like 10cm close).  This was a major letdown and it took us well into the hack to realize that we were going to be unable to use the devices in this manor.





TI Sensor Tag

I have now ordered the "estimote" beacons, with the hopes that you will work with the above idea. I have already had a few reports that they are higher powered and should not suffer from the same shortcomings. Here is a link to the product http://estimote.com

But like any good hacker you adapt and move on.  We changed the idea from trying to position a moving target to more of a check-in system.  By that I mean you would bring your device right up to the tag.. once we knew you were really close to the device we assigned you the same position as the fixed tag.  From here we displayed your location and the locations of "points of interest".

Once you know there are points of interest you can switch into the AR mode that we have and view information about the objects right on top of the camera.  You could imagine that you have entire product schema that you could drill down into to learn more, or even overlay video or virtual message boards that people could write on ect.

In the end the project lacked the right visuals to really impress the crowd.  However the knowledge gained was really valuable and I will be moving forward with this in the future.

Links to some source code on github

android-scala-ble - this is the scanning code required for android to locate devices and their signal strength.

android-scala-gl - OpenGL ES 2 library to help drawing basic shapes and sprites for the Augmented Reality portion of the project.

Introduction to Currying in Scala

Scala and functional programming

I have recently moved to programming in Scala. I started with 1 book and have now completed Functional Programming Principles in Scala , a free course on coursera. I am also part way into the next course on Principles of Reactive Programming.So far I have been having a ton of fun, and been blown away at the power of functional programming. I wanted to share some notes that helped me to understand some of the topics that I have studied.

Introduction to Currying in Scala

What is Currying
Currying is named after "Haskell Brooks Curry" a mathematician and logician.
Looking at A Tour of Scala: Currying: "Methods may define multiple parameter lists. When a method is called with a fewer number of parameter lists, then this will yield a function taking the missing parameter lists as its arguments."
Currying is a way of applying partial functions in an effort to make your code operate or appear more like built in language constructs thus making it more readable.

Here is some text from the book "Programming in Scala: A Comprehensive Step-by-Step Guide"
Currying allows you to make new control abstractions that feel like native language.... and, A curried function is applied to multiple arguments lists instead of just one.

Partial Functions
To better understand Currying we must take a look at partial functions. Functions in Scala and other functional languages are first class objects. These objects are generated for you by the compiler and contain a method apply. When you call a function

def f = {}
f()        // you are actually calling f.apply()

One of the things we can do with a function that contains a number of arguments is to supply "some" of the arguments. The compiler will again generate a function for you but this time it will also generate a wrapper with the partially applied values. Lets take a look at this in a Scala worksheet.

def mul3( a: Int, b: Int, c:Int ): Int = {
 a * b * c
}                                         // mul3: (a: Int, b: Int, c: Int)Int
mul3( 2, 2, 2)                            // res0: Int = 8

val partial = mul3( 5, _: Int , 5 )       // partial  : Int => Int = >function1<

Note that the compiler gave us a >function1<. This is a function that takes a single parameter, in our case the middle parameter to the original function that we left as a place holder. What was actually generated would be something similar to the following.

def f1( b: Int ): Int = {
   mul3(5, b, 5)
}

Currying
So now that we have some background lets take a look at a curried function. We will write a function prod and then show the equivalent curried function curriedProd

def prod( a: Int, b: Int) = { a * b }     //> prod: (a: Int, b: Int)Int

def curriedProd(a: Int)( b: Int) = { a * b }
                                                  //> curriedProd: (a: Int)(b: Int)Int

When you envoke the curried function you actually get another function with one of the arguments applied to it. This can be seen by supplying a place holder to the curried function as follows

val timesThree = curriedProd(3)_          //> timesThree  : Int => Int = 
 
val result = timesThree(3)                //> result  : Int = 9                                                  //> curriedProd: (a: Int)(b: Int)Int

Now that we have seen how currying works. We can now look at a number of uses.

Function Calling Syntax
Lets again define a simple function. This function takes a single parameter and adds five to its value. There are actually 2 different function calling syntax (there are actually more then 2). One that uses the normal parenthesis for arguments () and another method that uses curly braces or block syntax {}. Lets take a look at an example

def plusFive( x: Int ) = {
  x + 5
 }                                         //> plusFive: (x: Int)Int
 
val b = plusFive(10)                      //> b  : Int = 15
val c = plusFive{ 10 }                    //> c  : Int = 15

The purpose of substituting curly braces for parenthesis is to enable client programmers to write function literals between curly braces. One thing to note however is that we can only supply this kind of syntax if the function has a single argument. But wait !! Using currying it is always possible to break arguments out into their own argument list, thus yielding a function with a single parameter. Here is a small example.

def applyOpt( a: Int, b: Int, op: (Int, Int) => Int) = {
  op( a, b )
 }                                         //> applyOpt: (a: Int, b: Int, op: (Int, Int) => Int)Int
 
 applyOpt( 3, 3, (a: Int, b: Int) => { a * b } )
                                                  //> res2: Int = 9
 
 
 
 def applyOpt2( a: Int, b: Int)(op: (Int, Int) => Int) = {
  op( a, b )
 }                                         //> applyOpt2: (a: Int, b: Int)(op: (Int, Int) => Int)Int
 
 
 applyOpt2( 4, 4 ){ (a,b) => a*b }         //> res3: Int = 16

// or even..

applyOpt2( 4, 4 ){ _ * _ }                //> res3: Int = 16

Now that we have seen how this works it is time to put it all together in a real world example.

The Loan Pattern

The loan pattern is a way of lending resources to the caller while managing the lifecycle of those resources when they are no longer used. Coming from a C# background this is exactly the purpose of the "using" key word in C# .net

Lets finish off with a concrete example of this pattern in action.

def withOutput(f: OutputStream => Any) {
  val out: OutputStream = getOutputStream()
 
  try {
    f(out)
    out.flush()
  }
  finally {
    out.close() 
  }
}

// Now we can write...
withOutput { out => out.write(response.getBytes()) }

Android SDK Setup for Linux Mint

Quick Setup reference:
You will need to get the correct sun-java package. There are a few ways to do this. Perhaps the easiest is to update your apt repo with one of the links listed in this article here. If you choose to install the package via the oracle website (which is what I did) you will Require the 32 bit version for android. There is an overview of how to install this on linuxmint.com.

Next make sure that you download the correct SDK bundle from the android website. Download and uncompress to the folder of your choosing.

Configuring your device will require that you know your vendor id. Make sure your device is attached via USB and use the command lsusb. This will give you the vendor id. Once you have down this you should be able to add it to your /etc/udev/rules.d/51-android.rules. You will need to edit the file as root.

Here is an example configuration.
# adb protocol on passion (Nexus One) SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0600" 

Once you have this line added you will need to restart you ADB server
/sdk/platform-tools/adb kill-server
/sdk/platform-tools/adb start-server 
It might server to add a symlink to adb so you can execute it anywhere in your environment.

Lastly if you are still having trouble with the device not showing up reference this post.
Your device still can't display? Make sure your "Project Build Target" Android version is supported in your Device.

• Check your device's Android version. In your device, select Settings->About Device.
• Check Android version of your project. Right click your project->Properties->Android->ProjectBuildTarget.
• Make sure that it's not newer than your device's version.

Android SCALA

For creating android application with Scala (Which is more fun) download the scala IDE.

Follow all the documentation and you should be good to go.