Thursday, September 4, 2014

Example of Beegae - Part 3

Now we're finally getting to the code. This app maintains a list of names, so naturally I made a struct called "Name". Here is models/model.go:

  // model
package models

import (
    "appengine"
    "appengine/datastore"
)

type Name struct {
    FirstName, LastName string
}

const COLLECTION = "name"

func (this Name) String() string {
    return this.FirstName + " " + this.LastName
}

func (name Name) GetMatchCount(c appengine.Context) (count int, err error) {
    //    c := appengine.NewContext(rq)
    q := datastore.NewQuery(COLLECTION).Filter("FirstName =", name.FirstName).
        Filter("LastName =", name.LastName)
    var results []Name
    _, err = q.GetAll(c, &results)
    if err != nil {
        count = -1
    } else {
        count = len(results)
    }
    return
}

func (this Name) Add(c appengine.Context) error {
    key := datastore.NewIncompleteKey(c, COLLECTION, nil)
    _, err := datastore.Put(c, key, &this)
    return err
}

func GetAllNames(c appengine.Context) (names []Name, err error) {
    q := datastore.NewQuery(COLLECTION)
    _, err = q.GetAll(c, &names)
    return
}

As you can see, I'm using Datastore. The COLLECTION constant sets the data type string. The GetMatchCount function queries the database for names matching the incoming name, then calculates the number of matches and returns the total. The GetAll API function retrieves all of the documents matching the query. NewIncompleteKey generates a document key. Lastly, the String function tells Go how to convert a Name to a string. We will see how that is used soon. The Contexts determine which database gets accessed. 

Now we'll get to the controllers. Here is controllers/default.go:

package controllers
import (
    "fmt"
    "github.com/astaxie/beegae"
    "log"
    "models"
)

type MainController struct {
    beegae.Controller
}

func (this *MainController) Get() {
    this.Data["Title"] = "Beegae Example #1"
    this.TplNames = "form.tpl"
}

type VerifyController MainController

func (this *VerifyController) Post() {
    context := this.AppEngineCtx
    first := this.GetString("first")
    last := this.GetString("last")
    name := models.Name{first, last}
    count, err := name.GetMatchCount(context)
    message := fmt.Sprintf("%s %d", name, count)
    if err != nil {
        message = err.Error()
    }
    if count == 0 {
        if err := name.Add(context); err != nil {
            message = err.Error()
        }
    }
    this.Data["Title"] = "Beegae Example #1"
    this.Data["Message"] = message
    this.TplNames = "verify.tpl"
}

type ListController struct {
    beegae.Controller
}

func (this *ListController) Get() {
    context := this.AppEngineCtx
    //    var names []models.Name
    names, err := models.GetAllNames(context)
    if err != nil {
        log.Fatal(err.Error())\
    }
    this.Data["Title"] = "Beegae Example #1"
    this.Data["Names"] = names
    this.TplNames = "list.tpl"
}

GetString retrieves the form variable with the given name and casts it as a string. As you can see, the Sprintf function refers to a variable of type "Name" as a string. models.String tells Go how to implicitly cast that structure. In Beegae, constructors are actually types, which are used as receivers for the appropriate REST function. 
The views are pretty much trivial except for list.tpl. Here that is:

  
<!DOCTYPE html>

<html>
      <head>
        <title>{{.Title}}</title>
        <link rel="stylesheet" href="/static/css/style.css" type="text/css" />

    </head>
    <body>
        <h3>Updated List</h3>
        <ul>
            {{range $v := .Names}}
                <li>{{$v}}</li>
            {{end}}
        </ul>
    </body>
</html>        

The "range" keyword loops through the list of names and assigns each entry in turn to $v. As you can see, the data is treated like a string. The routing is fairly simple:

package routers
import (
    "beegaenames/controllers"
    "github.com/astaxie/beegae"
)

func init() {
    beegae.Router("/", &controllers.MainController{})
    beegae.Router("/verify", &controllers.VerifyController{})
    beegae.Router("/list", &controllers.ListController{})
}

That is routers/myrouter.go. Each Router has a URL and a variable. As you saw in earlier code, those variables are usually structs that contain a member of type beegae.Controller. The last thing we'll look at is how I deployed this app.

Friday, August 29, 2014

Example of Beegae - Part 2

The next step was to establish and configure my project. Before we get into that, I wanted a couple of Go packages to make things easier, namely github.com/astaxie/beego and github.com/beego/bee. I compiled the second package into the "bee" command.
I already had a directory in my root called "goprojects2". I created a "src" subdirectory. Then I went into the SDK command prompt and issued:

set gopath=%gopath%;c:\goprojects2

then went to the goprpjects2 directory and entered

bee new beegaenames

The "bee" command created a scaffold for my project. However, I had to convert the project from Beego to Beegae. I went into main.go (in goprojects2/src/beegaenames), and replaced "beego" with "beegae" .  I also went into controllers/default.go and routers/router.go, replacing "beego" with "beegae".
I needed to do one last thing before writing Go, namely create index.yaml on the same level as main.go. This file indexed my database. Here are the contents:

indexes:

- kind: Name
  ancestor: no
  properties:
  - name: FirstName
    direction: asc
  - name: LastName
    direction: asc

In the next installment, we'll get into the actual code.

Tuesday, August 26, 2014

Example of Beegae (Beego for App Engine) - Part 1

Sorry I've been gone for so long! I recently created a simple web form using Beegae. The setup was rather involved, but worth it. I will assume you have Git and Python 2.x already installed. I needed to install Go (http://golang.org/) and Google Cloud SDK (https://cloud.google.com/developers/ ; click "Download the SDK" for instructions). I used Datastore, the original NoSQL native to App Engine.

The next thing I had to do was install the Go SDK. To to that, I went into the Google Cloud SDK console window, and issued

gcloud components update gae-go

If you're using the Windows version, there might be a bug in goapp.bat. You will probably find it at

C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine

I issued the following to retrieve the Beegae Go package:

goapp get github.com/astaxie/beegae

and got this error...

'C:\Program' is not recognized as an internal or external command,
operable program or batch file.

I fixed the problem by these steps:

  1. Open Notepad as Administrator
  2. Open goapp.bat
  3. Put the %GOROOT%\bin\%EXENAME% line, along with all the parameters, on a new line
  4. Put quotation marks (") around %GOROOT%\bin\%EXENAME%
  5. Save (make sure NOT as a .TXT file) and Exit
If you're a bit squeamish, Copy goapp.bat as goapp.bat.old first. I reissued the above "goapp" command, and this time it ran. In the next installment, I'll get into how I configured this project.



Monday, February 10, 2014

After a long "vacation"...

Hi all!

I'm sorry I left you all in the lurch. I have been busy writing code, and you can see some of it at https://github.com/ozonesurfer . As a warning, Tiedot is a work in progress, and if you try to use my current code with Tiedot 2.0, it won't work.

That being said, I thought I would put my  2 cents in on some of the current "wars of religion". Like Tiedot, the tech world is in transition. Transitions are messy things; you have to phase the old out gracefully, and allow the new to mature. People are ditching their landlines for cell phones. People now text instead of call, and e-mail/post instead of writing letters. I'm not going to say one set of technology is better than others. They're just different, each with their benefits and flaws.

As for what I've seen, one of my friends bought a Chromebook and a Windows Phone, and she's perfectly happy with both. Another friend feels incapable of using Windows. She has an old iMac, and plans to eventually get a new one.

 As for me, I bought this Vista HP laptop in 2009; now I use Windows 8.1 on the same machine. My only real problem is my graphics chips sometimes struggle with video. And yes, I once used Windows 7 too. The only real complaint I have with Windows 8.x is the lack of built-in instructions. Despite that shortcoming, I leveraged my knowledge of previous versions and my brains to figure out how to use it. I demoed this knowledge to a couple friends, and not only did they love what they saw, one experimented in front of me and we learned more features together... another later bought a Windows 8 phone.

I consider Windows 8.x as a product of the transition from computers as they were once known to tablets/smartphones. You can't please everyone, but programmers (like me) are stuck with having to try. Some people think change is bad (or at least too inconvenient), and others think change doesn't happen fast enough. I just work with what's available, and I try to keep an open mind to the new.

I think that's embodied in my current software stack: Windows, document-based NoSQL, and Go. I can hear you say "Whrere is the web server? What? No Apache?".  As for Apache or IIS, with Go I don't really need them. The Go framework comes with the "net/http" package, which has the ListenAndServe function. I can specify any web address/port I have permission to manipulate as a parameter.

I'm not saying LAMP is bad or useless. If you know what you're doing, it gets the job done. And yes, I've used JavaScript (namely jQuery) in Go websites. And with Gorilla's websockets, I can broadcast user input in a chatroom website, or even manipulate my websites using the command line. If anyone knows how to do the latter using something else, I'd love to read that code. However, the command line will have to be read, and the entered data be used by the website,while the website is running...no cheating! E-mail it to me at ross.albertson@yahoo.com.      

I haven't seen screenshots of Windows 8.1 Update, but I'm not freaking out about it. I figured out how to use 8.0 and 8.1, and with God's help I'll learn the next version. And I'm not married to Go. I plan to learn Java 8 (or try to) fairly soon. If you stop learning, you stop growing as a person. Besides, to me a technological occupation means signing up to be a professional student to some degree. One last thing... one's belief in a higher power is sacred, but no technology is... if something you created can't be changed, it's broken.

Tuesday, September 24, 2013

Go (golang), MVC and MongoDB, part 2

Now comes the fun part, the actual code. The first thing we need to do is configure the website. That consists of routes.go and config.go, both in src/gomgoweb. First we'll tackle the routes. Here is that file in full:
routes.go
// routes
package gomgoweb

import (
// "fmt"
"github.com/QLeelulu/goku"
)

// routes
var Routes []*goku.Route = []*goku.Route{
// static file route
&goku.Route{
Name:     "static",
IsStatic: true,
Pattern:  "/static/(.*)",
},
// default controller and action route
&goku.Route{
Name:    "default",
Pattern: "/{controller}/{action}/{id}",
Default: map[string]string{"controller": "home", "action": "index",
"id": "0"},
},
}

This file configures the two kinds of web addresses we'll be using. The "static" route is for things like JPEG files,or, in our case, a CSS file. The "default" route is for the MVC web addresses. As you probably guessed, the Default dictionary determines the home address of the website (/home/index).

Noe for the more difficult part, config.go. This contains the domain name of the website, as well as the site's behavior and global constants. Here is the whole thing:

config.go
// config
package gomgoweb

import (
"github.com/QLeelulu/goku"
"path"
"runtime"
"time"
)

var Config *goku.ServerConfig = &goku.ServerConfig{
Addr:           "localhost:8080",
ReadTimeout:    10 * time.Second,
WriteTimeout:   10 * time.Second,
MaxHeaderBytes: 1 << 20,
StaticPath:     "static",
ViewPath:       "views",
LogLevel:       goku.LOG_LEVEL_LOG,
Debug:          true,
}

const (
DATABASE     = "gomgoweb-8"
BAND_COL     = "bands"
LOCATION_COL = "locations"
GENRE_COL    = "genres"
ALBUM_COL    = "albums"
DATABASE_DSN = "localhost"
)

func init() {
// project root directory
_, filename, _, _ := runtime.Caller(1)
Config.RootDir = path.Dir(filename)
}

You could use just about any string for the database name. "COL" here is short for "collection". In MongoDB, data is first organized into databases, then into collections. "Addr" determines the domain name. In a real life website, this would more likely be a publicly accessed web address. This file also tells Goku where to find the view HTML files, as well as gives a potential alias for the the "static" directory.  

We of course need a main package and entry point. Those are provided by gomgoweb.go, which goes in the project's root directory. Here it is:

gomgoweb.go
// gomgoweb
package main

import (
// "fmt"
"github.com/QLeelulu/goku"
"gomgoweb"
"gomgoweb/controllers"
// "gomgoweb/models"
// "labix.org/v2/mgo/bson"
"log"
)

func main() {
// fmt.Println("Hello World!")
rt := &goku.RouteTable{Routes: gomgoweb.Routes}
middlewares := []goku.Middlewarer{}
s := goku.CreateServer(rt, middlewares, gomgoweb.Config)
goku.Logger().Logln("Server start on", s.Addr)
log.Fatal(s.ListenAndServe())
}

var home = controllers.HomeController
var band = controllers.BandController
var album = controllers.AlbumController

You can safely ignore the commented-out lines. As you can see, the main() function creates the web server in memory and connects it to the files we previously created. You also see that this file establishes the controllers as global variables. The names of those variables can be anything that's legal in Go. I did things this way for the sake of simple explanation. You could have created those variables in the controller files, given them empty ("_") names, and imported the controller package as _ "gomgoweb/controllers", but I think the way I coded it is more intuitive. The ListenAndServe() method is what actually starts the web server up.

At this point, if you left out all mention of controllers, this code would run. However, you wouldn't be able to view the results in your browser. For that, you need a controller and HTML template file. We'll create those soon. In the next installment, we'll get into other global aspects of this project: the CSS and the layout template.  

Monday, September 23, 2013

Go (golang) and MongoDB Example 2 - A Series on Goku (MVC) Websites

Sorry I've been gone for so long. I decided to discontinue the Couchbase series in favor of MongoDB. I'm going to assume you have a working copy of Go and MongoDB. If you don't, you can get those at http://golang.org/ and http://www.mongodb.org/ . You will also need http://bazaar.canonical.com/en/ and Git (http://git-scm.com/ or https://github.com/) to install the necessary Go packages. You can retrieve those packages by issuing the following commands:

go get github.com/QLeelulu/goku

go get labix.org/v2/mgo

You will also need the /data/db path (C:\data\db in Windows) in your root for MongoDB to work.

This website/web server does not need IIS or Apache to run. It basically catalogs a CD collection. I used the LiteIDE (https://code.google.com/p/golangide/downloads/list) environment to write this program. Zeus (http://www.zeusedit.com/go.html) is another possibility, although you have to pay for it after 45 days.

Another piece of housekeeping is the directory tree. You should create a "gomgoweb" directory on your machine, and you'll need the following paths under it (use "\" instead of "/" if you're using Windows):

src/gomgoweb
src/gomgoweb/controllers
src/gomgoweb/models
src/gomgoweb/views/home
src/gomgoweb/views/band
src/gomgoweb/views/album
src/gomgoweb/views/shared
src/gomgoweb/static

If you have any experience with the MVC architecture, this tree should look familiar. Go expects to see all packages other than "main" under a src/<package_name> directory. The last thing you need to do (if they haven't been done for you) is set up the "gopath" environment variable. That should contain a path to your default Go package directory and the root path of this project. In Windows/DOS, that would look like

set gopath=C:\Go\pkg;C:\gomgoweb

Well, you get the idea. In the next installment, we'll get our hands dirty on some code.

Tuesday, April 16, 2013

Go (golang) And MongoDB - My First Example

As you probably know, Go is a language under development by Google and MongoDB is a database system created by 10gen. You can find instructions for installing Go (including for Windows, which I use) at http://golang.org/doc/install . You can also install MongoDB from http://www.mongodb.org/downloads . You will need to create a /data/db (Unix-based) or C:\data\db (Windows) directory structure for MongoDB to store your data.
You'll need other things to run this example. First, you'll need Bazaar's bzr command. You'll find what you need at http://wiki.bazaar.canonical.com/Download. Once you have that installed, go to a command prompt and enter

go install labix.org/v2/mgo

That should install the mgo (pronounced "mango") package. You might need to set the GOPATH environment variable first; I suggest setting it to the "pkg" subdirectory of Go's installation path. GOPATH tells Go where to expect/install packages. Setting it to Go's installation directory itself isn't always a good idea. I learned that the hard way. Labix's installation script uses the bzr command; that's why you need Bazaar.
Now you're ready to do some coding. This example will show you how to retrieve first and last names from the user (it's a console app), check the database to see if that name was already entered, save the user's input if it wasn't, and finally display all the names in the MongoDB collection. For those not used to document-based NoSQL, a "collection" is similar to a table; "documents" are similar to records. The principal difference is documents have no set structure; there is no schema forcing them to have anything is common except having an ID. Some document-based systems might organize your documents for you based on their "plain-old" (POJO/POCO/etc) class. Also, some (like CouchDB) expect to see other fields, like a revision code.

Anyway, on to our code. The first things you do in a Go program file are declaring a package name and retrieving the packages your file needs. In our case, that would be done as follows...


package main

import (
"fmt"
"labix.org/v2/mgo"
"bufio"
"os"
"strings"
"labix.org/v2/mgo/bson"
)

The "fmt", "bufio", and "os" packages handle the console I/O. As you might expect, the "strings" package has the methods for string-handling. The rest are for interacting with MongoDB. Go doesn't have classes in the usual sense, just structures. Unlike in C/C++ however, you don't need header files. You can also attach functions to Go structures easily, as you will see. Here are the structures and the function we will need:


type Name struct {
Id bson.ObjectId `bson:"_id"`
MyName QueryName
}

type QueryName struct {
FirstName string `bson:"FirstName"` 
LastName string `bson:"LastName"`
}

func (q *QueryName) ConvertToInterface() (interface{}) {
return  map[string]interface{}{
"FirstName": q.FirstName,
                        "LastName": q.LastName,
                }
}  



I did things this way to reduce code duplication; you could specify first-name and last-name fields in the Name structure directly instead. The Id field as written is necessary for creating MongoDB documents. Other parts of MongoDB tend to misbehave without it. The  `bson:` clauses tell mgo how to serialize the Go fields into BSON format, which is how MongoDB stores data. Without it, that field won't be stored. I created two structures because sending MongoDB queries structures with Ids can cause problems. The ConvertToInterface function translates the QueryName structure into a string-indexed "array" that MongoDB queries can use. The "q" pointer works like the "this" pointer in object-oriented languages; it also attaches that function to the QueryName structure.
The next thing is to start creating the entry point function and connect to the database. Assuming that you're running the code on the same machine as MongoDB, that would be


func main() {
session, sessionErr := mgo.Dial("localhost")
if sessionErr != nil {
fmt.Println(sessionErr)
} else {
fmt.Println("Session created")

Once you realize Go functions can return multiple values, that should be self-explanatory. The Dial function connects the code to MongoDB. The := operator initializes variables. In Go, variables can be defined implicitly by that operator. Once a variable is initialized, you use the = operator.
Now, to ask for and read the user's input, you would use


r :=bufio.NewReader(os.Stdin)
fmt.Print("First name: ")
first, _ := r.ReadString('\n')
fmt.Print("Last name: ")
last, _ := r.ReadString('\n')
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)

This is the best way to read the keyboard in a console app if the input can contain spaces. If you've used older C-style languages, os.Stdin should look familiar as the keyboard buffer. ReadString('\n') gets the input as a string that ends when the user hits the Enter key. Unfortunately, it includes the newline in the string. TrimSpace strips that character off.
Now, it's time to finish connecting to MongoDB and look up the input:


database := session.DB("go_mongo")
collection := database.C("names")
/* collection.DropCollection()
collection = database.C("names") */
nameForQuery := QueryName{FirstName: first, LastName: last}
// query1 := collection.Find(bson.M{"myname": bson.M{"FirstName": first, "LastName": last}})
param := nameForQuery.ConvertToInterface()
query1 := collection.Find(bson.M{"myname": param})
count, _ := query1.Count()

If you haven't guessed, "go-mongo" is the name of the database, and "names" is the collection's name. If you're familiar with C-style languages, you'll probably recognize the comments. The commented code can be omitted. If you're curious, DropCollection destroys the collection. As you can see, you can create a new version of the collection after destroying it. Both Find function calls will work; they feed a first and last name to MongoDB to query for exact matches. As you might have guessed, the Count function retrieves the number of matches. It also optionally returns an error code, which you would check against nil to detect actual errors. To add the data you would do something like


name := Name{Id: bson.NewObjectId(), MyName: QueryName{FirstName: first, LastName: last}}
add_err := collection.Insert(name)
if add_err != nil {
fmt.Println("Error on add:", add_err)
} else {
fmt.Println("Name was successfully added")
}

"name" is a structure of type "Name", initialized with the entered data and a newly generated document Id. Any such structure could be added to the collection. If your domain structure is annotated like mine, with an identical Id field, it should work in your own code. Substructures and arrays are perfectly acceptable; those are some of the beauties of NoSQL.    
Now for the fun part, namely dumping a whole collection. Don't let other people's documentation get you down; it can be done with some trickery. Here is how I did it...

         var results []Name
collection.Find(nil).All(&results)
for _, name := range results {
fmt.Println(name.Id, ":", name.MyName.FirstName, name.MyName.LastName)
}

The ampersand denotes a by-reference argument. That is another way a Go function can return values. As you can see, I'm querying on "nil", that is to say, essentially nothing! That will retrieve the whole collection. The same trick works with CouchDB/Couchbase; it might work with other document-based systems. If your whole collection is nothing but Names, this code will work. If you mixed document types in the collection (which you can do), something like this might be necessary...




var results2 []bson.M
                collection.Find(nil).All(&results2)
for _, obj := range results2 {
fmt.Println(obj)
}

The for... range syntax is how Go does a for-each. "bson.M" is mgo's default document structure; it works like a string-indexed array (map).  Lastly, you close the connection as follows...


session.Close()
}
}

To run the Go interpreter, simply use the syntax go run program.go .  To compile a Go program, run go build program.go .  Go scripts always have the extension ".go". In Windows, "go build" creates an .exe file if successful. "go build -o filename" enables you to use a target name other than the default. Instead of ".dll", Go uses ".a" for libraries.

I hope you found my code useful. Check out my other blogs too. Bye for now, and God bless!