Simple and Powerful ReverseProxy in Go

In this article we will learn about reverse proxy, where to use it and how to implement it in Golang.

A reverse proxy is a server that sits in front of web servers and forwards client (e.g. web browser) requests to web servers. They give you control over the request from clients and responses from the servers and then we can use that to leverage benefits like caching, increased security and many more.

Before we learn more about reverse proxy, lets quickly understand the difference between a normal proxy (aka forward proxy) and reverse proxy.

In Forward Proxy, proxy retrieves data from another website on the behalf of original client. It sits in front of a client (your browser) and ensures that no backend server ever communicates directly with the client. All the client requests go through the forward proxy and hence the server only communicates with that proxy (assuming proxy is its client). In this case, the proxy masks the client.

Forward Proxy Flow

On the other hand, a Reverse Proxy sits in front of backend servers and ensures that no client ever communicates directly with the servers. All the client requests go to server via reverse proxy and hence client is always communicating to reverse proxy and never with the actual server. In this case, the proxy masks the backend servers. Few examples of reverse proxy are Nginx Reverse proxy, HAProxy.

Reverse Proxy Flow

Reverse Proxy Use cases

Load balancing: a reverse proxy can provide a load balancing solution which will distribute the incoming traffic evenly among the different servers to prevent any single server from becoming overloaded

Preventing security attacks: since the actual web-servers never need to expose their public IP address, attacks such as DDoS can only target the reverse proxy which can be secured with more resources to fend off the cyber attack. Your actual servers are always safe.

Caching: Let’s say your actual servers are in a region far from your users, you can deploy regional reverse proxies which can cache content and serve to local users.

SSL encryption: As SSL communication with each client is computationally expensive, using a reverse proxy it can handle all your SSL related stuff and then freeing up valuable resources on your actual servers.

Golang Implementation

package main

import (
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
)

// NewProxy takes target host and creates a reverse proxy
func NewProxy(targetHost string) (*httputil.ReverseProxy, error) {
	url, err := url.Parse(targetHost)
	if err != nil {
		return nil, err
	}

	return httputil.NewSingleHostReverseProxy(url), nil
}

// ProxyRequestHandler handles the http request using proxy
func ProxyRequestHandler(proxy *httputil.ReverseProxy) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		proxy.ServeHTTP(w, r)
	}
}

func main() {
	// initialize a reverse proxy and pass the actual backend server url here
	proxy, err := NewProxy("http://my-api-server.com")
	if err != nil {
		panic(err)
	}

	// handle all requests to your server using the proxy
	http.HandleFunc("/", ProxyRequestHandler(proxy))
	log.Fatal(http.ListenAndServe(":8080", nil))
}

And yes! Thats all it takes to create a simple reverse proxy in Go. We used standard library “net/http/httputil” and created a single host reverse proxy. Any request to our proxy server is proxied to the backend server located at http://my-api-server.com. The code is pretty much self-explanatory if you are from Go background.

Modifying the response

HttpUtil reverse proxy provides us a very simple mechanism to modify the response we got from the servers. This response can be cached or changed based on your use cases. Let’s see how we can make this change.

// NewProxy takes target host and creates a reverse proxy
func NewProxy(targetHost string) (*httputil.ReverseProxy, error) {
	url, err := url.Parse(targetHost)
	if err != nil {
		return nil, err
	}

	proxy := httputil.NewSingleHostReverseProxy(url)
	proxy.ModifyResponse = modifyResponse()
	return proxy, nil
}

func modifyResponse() func(*http.Response) error {
	return func(resp *http.Response) error {
		resp.Header.Set("X-Proxy", "Magical")
		return nil
	}
}

You can see in modifyResponse method, we are setting a custom header. Similarly you can read the response body, make changes to it, cache it and then set it back for the client.

In ModifyResponse you can also return an error (if you encounter it while processing response) which then will be handled by proxy.ErrorHandler. ErrorHandler is automatically called if you set error inside modifyResponse.

// NewProxy takes target host and creates a reverse proxy
func NewProxy(targetHost string) (*httputil.ReverseProxy, error) {
	url, err := url.Parse(targetHost)
	if err != nil {
		return nil, err
	}

	proxy := httputil.NewSingleHostReverseProxy(url)
	proxy.ModifyResponse = modifyResponse()
	proxy.ErrorHandler = errorHandler()
	return proxy, nil
}

func errorHandler() func(http.ResponseWriter, *http.Request, error) {
	return func(w http.ResponseWriter, req *http.Request, err error) {
		fmt.Printf("Got error while modifying response: %v \n", err)
		return
	}
}

func modifyResponse() func(*http.Response) error {
	return func(resp *http.Response) error {
		return errors.New("response body is invalid")
	}
}

Modifying the request

You can also modify the request before sending it to the server. In below example we are adding a header before sending it to server. Similarly, you can make any changes to the request before sending.

// NewProxy takes target host and creates a reverse proxy
func NewProxy(targetHost string) (*httputil.ReverseProxy, error) {
	url, err := url.Parse(targetHost)
	if err != nil {
		return nil, err
	}

	proxy := httputil.NewSingleHostReverseProxy(url)

	originalDirector := proxy.Director
	proxy.Director = func(req *http.Request) {
		originalDirector(req)
		modifyRequest(req)
	}

	proxy.ModifyResponse = modifyResponse()
	proxy.ErrorHandler = errorHandler()
	return proxy, nil
}

func modifyRequest(req *http.Request) {
	req.Header.Set("X-Proxy", "Simple-Reverse-Proxy")
}

Complete code

package main

import (
	"errors"
	"fmt"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
)

// NewProxy takes target host and creates a reverse proxy
func NewProxy(targetHost string) (*httputil.ReverseProxy, error) {
	url, err := url.Parse(targetHost)
	if err != nil {
		return nil, err
	}

	proxy := httputil.NewSingleHostReverseProxy(url)

	originalDirector := proxy.Director
	proxy.Director = func(req *http.Request) {
		originalDirector(req)
		modifyRequest(req)
	}

	proxy.ModifyResponse = modifyResponse()
	proxy.ErrorHandler = errorHandler()
	return proxy, nil
}

func modifyRequest(req *http.Request) {
	req.Header.Set("X-Proxy", "Simple-Reverse-Proxy")
}

func errorHandler() func(http.ResponseWriter, *http.Request, error) {
	return func(w http.ResponseWriter, req *http.Request, err error) {
		fmt.Printf("Got error while modifying response: %v \n", err)
		return
	}
}

func modifyResponse() func(*http.Response) error {
	return func(resp *http.Response) error {
		return errors.New("response body is invalid")
	}
}

// ProxyRequestHandler handles the http request using proxy
func ProxyRequestHandler(proxy *httputil.ReverseProxy) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		proxy.ServeHTTP(w, r)
	}
}

func main() {
	// initialize a reverse proxy and pass the actual backend server url here
	proxy, err := NewProxy("http://my-api-server.com")
	if err != nil {
		panic(err)
	}

	// handle all requests to your server using the proxy
	http.HandleFunc("/", ProxyRequestHandler(proxy))
	log.Fatal(http.ListenAndServe(":8080", nil))
}

Reverse proxy is very powerful and can be used for multiple use cases as explained above. You can try customising it as per your case and if you face any issues, I will be very happy to help you with that. If you found the article interesting, please share it so that it can reach to other gophers! Thanks a lot for reading.

Tagged : / /

Golang gRPC communication made easy!

In this post we are going to build a simple gRPC client and server application. I was recently trying to set this up in one of my projects and was struggling to do it. Hence I decided to write a post which focuses only on the exact steps to setup gRPC communication between server and client written in Golang.

I will not cover the basics of gRPC and why we should use it. If you are new to it you can read about it here. Before we start here are the pre-requisites:

Create project directory and initialise Go modules

Don’t worry if you are unaware with some of the files in below tree structure, for now just create empty files / directories using the given commands.

$ mkdir exchange-app
$ cd exchange-app
$ go mod init bank/exchange-app
$ mkdir proto
$ touch server.go client.go proto/currency_conversion.proto

$ tree .
.
├── client.go
├── go.mod
├── proto
│   └── currency_conversion.proto
└── server.go

Install gRPC and protoc-gen-go

$ go get -u google.golang.org/grpc
$ go get -u github.com/golang/protobuf/protoc-gen-go

Install protobuf

# Linux 
$ apt install -y protobuf-compiler
$ protoc --version 

# Mac OS
$ brew install protobuf
$ protoc --version 

For other operating systems, please refer this link.

Let’s setup the client and server communication

Define a service in a .proto file

The first step is to complete the currency_conversion.proto file which defines the gRPC service and the request and response messages using protocol buffers. Now open the currency_conversion.proto file in a text editor and paste the below content in it.

// currency_conversion.proto
// this file will be used to generate the grpc client and server interfaces in Go

syntax = "proto3";

package proto;

message CurrencyConversionRequest {
    string from = 1;
    string to = 2;
    float value = 3;
}

message CurrencyConversionResponse {
    float converted_value = 1;
}

service CurrencyConversionService {
    rpc Convert(CurrencyConversionRequest) returns (CurrencyConversionResponse) {}
}

Generate server and client code using the protocol buffer compiler

From the project root directory, run the below command to generate the gRPC client and server interfaces in Go which can be used as a Go package in the client and server code.

$ protoc -I proto/ proto/currency_conversion.proto --go_out=plugins=grpc:proto/

After executing the above command, if we again look at the tree structure of the current directory, we can see that a currency_conversion.pb.go file is generated.

$ tree .
.
├── client.go
├── go.mod
├── proto
│   ├── currency_conversion.pb.go // generated file
│   └── currency_conversion.proto
└── server.go

The generated file contains the following contents:

  • Golang struct type for each message defined in currency_conversion.proto file. It also has helper methods to get value of each field defined in the struct. As an example, look at the generated code for proto message CurrencyConversionRequest
type CurrencyConversionRequest struct {
    state         protoimpl.MessageState
    sizeCache     protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields

    From  string  `protobuf:"bytes,1,opt,name=from,proto3" json:"from,omitempty"`
    To    string  `protobuf:"bytes,2,opt,name=to,proto3" json:"to,omitempty"`
    Value float32 `protobuf:"fixed32,3,opt,name=value,proto3" json:"value,omitempty"`
}

func (x *CurrencyConversionRequest) GetFrom() string {
    if x != nil {
        return x.From
    }
    return ""
}

func (x *CurrencyConversionRequest) GetTo() string {
    if x != nil {
        return x.To
    }
    return ""
}

func (x *CurrencyConversionRequest) GetValue() float32 {
    if x != nil {
        return x.Value
    }
    return 0
}
  • An interface type which clients will use to call the methods implemented by the grpc server
type CurrencyConversionServiceClient interface {
    Convert(ctx context.Context, in *CurrencyConversionRequest, opts ...grpc.CallOption) (*CurrencyConversionResponse, error)
}

type currencyConversionServiceClient struct {
    cc grpc.ClientConnInterface
}

func NewCurrencyConversionServiceClient(cc grpc.ClientConnInterface) CurrencyConversionServiceClient {
    return &currencyConversionServiceClient{cc}
}

func (c *currencyConversionServiceClient) Convert(ctx context.Context, in *CurrencyConversionRequest, opts ...grpc.CallOption) (*CurrencyConversionResponse, error) {
    out := new(CurrencyConversionResponse)
    err := c.cc.Invoke(ctx, "/currency_conversion.CurrencyConversionService/Convert", in, out, opts...)
    if err != nil {
        return nil, err
    }
    return out, nil
}
  • An interface type which the servers will implement to serve the client requests
// CurrencyConversionServiceServer is the server API for CurrencyConversionService service.
type CurrencyConversionServiceServer interface {
    Convert(context.Context, *CurrencyConversionRequest) (*CurrencyConversionResponse, error)
}

// this is the function servers will use to register their struct object which implements the grpc service methods
func RegisterCurrencyConversionServiceServer(s *grpc.Server, srv CurrencyConversionServiceServer) {
    s.RegisterService(&_CurrencyConversionService_serviceDesc, srv)
}

Finish the server.go file

Now let’s implement the server.go

package main

import (
	"bank/exchange-app/proto"
	"context"
	"fmt"
	"net"

	"google.golang.org/grpc"
)

// myGRPCServer implements CurrencyConversionServiceServer interface
type myGRPCServer struct {
}

// actual implementation of the Convert method
func (s *myGRPCServer) Convert(ctx context.Context, request *proto.CurrencyConversionRequest) (*proto.CurrencyConversionResponse, error) {
	fmt.Printf("request received to convert currency %+v \n", request)

	// NOTE: here you can write the logic to convert the currency

	return &proto.CurrencyConversionResponse{
		ConvertedValue: 74.02,
	}, nil
}

func main() {
	fmt.Println("starting gRPC server application")

	// start a listener on the port you want to start the grpc server
	lis, err := net.Listen("tcp", ":9000")
	if err != nil {
		fmt.Println(err)
		return
	}

	// create a grpc server object
	srv := grpc.NewServer()

	// pass the address of the struct which implements the CurrencyConversionServiceServer interface
	proto.RegisterCurrencyConversionServiceServer(srv, &myGRPCServer{})

	// start the grpc server
	err = srv.Serve(lis)
	if err != nil {
		return
	}

	return
}

But what have we done here ? Let’s see what steps can be followed to write the server.go code

  1. Create a struct (give it any name you want). It should implement the CurrencyConversionServiceServer interface
// myGRPCServer implements CurrencyConversionServiceServer interface
type myGRPCServer struct {
}

// actual implementation of the Convert method
func (s * myGRPCServer) Convert(ctx context.Context, request *proto.CurrencyConversionRequest) (*proto.CurrencyConversionResponse, error) {
    fmt.Printf("request received to convert currency %+v \n", request)

    // NOTE: here you can write the logic to convert the currency

    return &proto.CurrencyConversionResponse{
        ConvertedValue: 74.02,
    }, nil
}

2. Start a listener on 9000 port (you can use port of your choice)

lis, err := net.Listen("tcp", ":9000")
if err != nil {
    fmt.Println(err)
    return
}

3. Using google.golang.org/grpc package’s NewServer method, create a new grpc server object. It creates a gRPC server object which has no service registered and has not started to accept requests yet

 srv := grpc.NewServer()

4. Register the defined struct to inform grpc about the object which implements the grpc server

proto.RegisterCurrencyConversionServiceServer(srv, & myGRPCServer{})

5. Start the gRPC server now which will listen for incoming requests at port 9000

// start the grpc server
err = srv.Serve(lis)
if err != nil {
    return
}

Finish the client.go file

Here is the implemented grpc client.go file and you can find the breakdown of each step after it.

package main

import (
	"context"
	"fmt"

	"bank/exchange-app/proto"

	"google.golang.org/grpc"
)

func main() {
	fmt.Println("starting grpc client application")

	// use grpc.Dial to create a grpc connection to running grpc server
	conn, err := grpc.Dial(":9000", grpc.WithInsecure(), grpc.WithBlock())
	if err != nil {
		fmt.Println(err)
		return
	}

	defer conn.Close()

	// use the NewCurrencyConversionServiceClient function defined in generated .pb.go file
	// to generate a grpc client object
	// this object can be used to call methods implemented in grpc server
	client := proto.NewCurrencyConversionServiceClient(conn)

	// create the grpc request
	request := &proto.CurrencyConversionRequest{
		From:  "INR",
		To:    "USD",
		Value: 1,
	}

	// call the grpc server method Convert using the client
	response, err := client.Convert(context.Background(), request)
	if err != nil {
		fmt.Println(err)
		return
	}

	// print the response
	fmt.Println("converted currency value is: ", response.GetConvertedValue())
}
  1. Dial a connection to the server
// use grpc.Dial to create a grpc connection to running grpc server
conn, err := grpc.Dial(":9000", grpc.WithInsecure(), grpc.WithBlock())
if err != nil {
    fmt.Println(err)
    return
}

2. Create a grpc client object using the connection object. This object will be used to call server methods

client := proto.NewCurrencyConversionServiceClient(conn)

3. Create the grpc request and call the server methods

// create the grpc request
request := &proto.CurrencyConversionRequest{
    From:  "INR",
    To:    "USD",
    Value: 1,
}

// call the grpc server method Convert using the client
response, err := client.Convert(context.Background(), request)
if err != nil {
    fmt.Println(err)
    return
}

And that’s it.

Time to test

  1. Run the GRPC server
$ go run server.go
starting gRPC server application

2. Run the client

$ go run client.go
starting grpc client application
converted currency value is:  74.02

When you run the client code, server will also show the log message

request received to convert currency from:"INR" to:"USD" value:1

That was it! I hope you enjoyed the post and learned how we can establish gRPC communication between client and server written in Golang. If you enjoyed the post, please like and share. Any feedbacks or suggestions will encourage me to write more. Thanks for reading!

Tagged : / / / / /

Simple and elegant Vim IDE setup for Go

I just love working in Vim and cannot even think of switching to a new editor. If you are also a Vim lover and you use Golang, you must read this post to make sure you have the right setup.

Here is how my vim looks like and yes the look is very simple though powerful.

Following are the most important features which I prefer to have:

  1. Run gofmt and goimports on save
  2. Peek at documentation
  3. Autocomplete
  4. Jump to symbols or functions definitions
  5. Automatic variable type and function signature info
  6. Pretty and lightweight status bar at bottom
  7. Version control information

In order to have all these features integrated in Vim, I used the following plugins:

vim-go development plugin

Installation
git clone https://github.com/fatih/vim-go.git ~/.vim/pack/plugins/start/vim-go
Uninstall
rm -r ~/.vim/pack/plugins/start/vim-go 

Lean and Mean status bar

Install

git clone https://github.com/vim-airline/vim-airline ~/.vim/pack/dist/start/vim-airline

Uninstall

rm -r ~/.vim/pack/dist/start/vim-airline

An awesome Git wrapper

Install

git clone https://github.com/tpope/vim-fugitive.git ~/.vim/pack/dist/start/vim-fugitive 

Uninstall

rm -r ~/.vim/pack/dist/start/vim-fugitive

After installing all the plugins, here is how you can configure ~/.vimrc file:

syntax on

colo pablo

set encoding=utf-8

set fileencoding=utf-8

set number

set shiftwidth=2

" settings specific to Go file types
autocmd Filetype go setlocal tabstop=4 shiftwidth=4 softtabstop=4

filetype plugin indent on

set backspace=indent,eol,start

" vim-go commands
let g:go_fmt_command = "goimports"
let g:go_auto_type_info = 1
let g:go_highlight_functions = 1
let g:go_highlight_methods = 1
let g:go_highlight_fields = 1
let g:go_highlight_types = 1
let g:go_highlight_operators = 1
let g:go_highlight_build_constraints = 1

I hope you have enjoyed this post and it will be helpful for you to setup your Vim. Please like and share and feel free to comment if you have any suggestions or feedback.

Tagged : / / / /

Simple HTTP request logging middleware in Go

When we’re building a web application there’s some shared functionality that we want to run for many (or even all) HTTP requests. We might want to log every request, gzip every response, or authenticate request before doing any processing. There are already libraries present which do this, but its always fun learning something new and building on your own.

In this post I’m going to explain how to make a simple custom request logging middleware which can be used with gorilla/mux. Our middleware will log each incoming request to the application and will log the following parameters:

  1. Request Method
  2. Request Host
  3. Request Path
  4. Request query parameters
  5. Time took to complete request
  6. Response status code

Given below is a simple Go program which has 2 APIs, /ping, /hello.

package main

import (
	"net/http"

	"github.com/gorilla/mux"
)

func main() {
	r := mux.NewRouter()
	r.HandleFunc("/ping", handlePing).Methods(http.MethodGet)
	r.HandleFunc("/hello", handleHello).Methods(http.MethodGet)
	http.ListenAndServe(":http", r)
}

func handlePing(rw http.ResponseWriter, req *http.Request) {
	rw.Write([]byte("I am working...\n"))
	return
}

func handleHello(rw http.ResponseWriter, req *http.Request) {
	rw.Write([]byte("Hi, how are you...\n"))
	return
}

Writing the middleware

Now we will write a simple middleware, which logs all the incoming requests to our application. Initially we will only focus on printing the first 4 parameters.

func RequestLoggerMiddleware(r *mux.Router) mux.MiddlewareFunc {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
            defer func() {
                log.Printf(
                    "[%s] %s %s %s",
                    req.Method,
                    req.Host,
                    req.URL.Path,
                    req.URL.RawQuery,
                )
            }()

            next.ServeHTTP(w, req)
        })
    }
}

We can use this with mux router as below:

r.Use(RequestLoggerMiddleware(r))

So the complete main function looks like

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/ping", handlePing).Methods(http.MethodGet)
    r.HandleFunc("/hello", handleHello).Methods(http.MethodGet)
    r.Use(RequestLoggerMiddleware(r))
    http.ListenAndServe(":http", r)
}

Lets try calling the APIs now and check how the requests are being logged now:

Perfect, this looks great! Now we also want to add the time taken by the API. This can be done very easily in the middleware function by adding the following:

func RequestLoggerMiddleware(r *mux.Router) mux.MiddlewareFunc {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
            start := time.Now()
            defer func() {
                log.Printf(
                    "[%s] [%v] %s %s %s",
                    req.Method,
                    time.Since(start),
                    req.Host,
                    req.URL.Path,
                    req.URL.RawQuery,
                )
            }()

            next.ServeHTTP(w, req)
        })
    }
}

So now the new output is something like below

Looks better! So the only part remaining is to add the response status code in the log message.

Do we have access to response status code in middleware ? Does http.ResponseWriter help ?

Frankly speaking, there is no direct way of adding that. In Go APIs, we use the response writer http.ResponseWriter object to write status codes in APIs. If we closely look at the implementation of http.ResponseWriter , it is an interface and has the following methods:

type ResponseWriter interface {
    Header() Header
    Write([]byte) (int, error)
    WriteHeader(statusCode int)
}

From our API methods, we call WriteHeader method on response writer object.

func handleHello(rw http.ResponseWriter, req *http.Request) {
    rw.WriteHeader(http.StatusOK)
    rw.Write([]byte("Hi, how are you...\n"))
    return
}

Though in our middleware we don’t have access to this status code and thus we cannot print it. To solve this problem, we will use a very cool feature of Golang and will embed the ResponseWriter interface in a custom struct. This custom struct will have a statusCode field inside it and thus can be used to print the status code in request logs. Let’s see how:

type statusResponseWriter struct {
    http.ResponseWriter
    statusCode int
}

// NewStatusResponseWriter returns pointer to a new statusResponseWriter object
func NewStatusResponseWriter(w http.ResponseWriter) *statusResponseWriter {
    return &statusResponseWriter{
        ResponseWriter: w,
        statusCode:     http.StatusOK,
    }
}

So we created a new structure statusResponseWriter which contains the statusCode field in it. Now since this structure embeds http.ResponseWriter interface, its methods are seen as if the custom struct is also implementing them.

When interfaces are embedded, their methods are promoted automatically. That means methods of interface are seen as if they are methods of the embedding struct

This means an object of our custom struct can be passed to ServeHTTP function, since it implements the http.ResponseWriter interface. But here comes another cool feature, we can also override the methods of interface.

It’s here where we do all the magic to assign the statusCode field of our custom struct whenever there is a WriteHeader from the API.

// WriteHeader assigns status code and header to ResponseWriter of statusResponseWriter object
func (sw *statusResponseWriter) WriteHeader(statusCode int) {
    sw.statusCode = statusCode
    sw.ResponseWriter.WriteHeader(statusCode)
}

Now let’s see how we can use the custom struct statusResponseWriter to print the status code in request logs. So instead of passing w http.ResponseWriter to ServeHTTP method, we pass sw statusResponseWriter to it. Then we use sw.statusCode field to log the response status code and thats it the work is done here!

func RequestLoggerMiddleware(r *mux.Router) mux.MiddlewareFunc {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
			start := time.Now()
			sw := NewStatusResponseWriter(w)

			defer func() {
				log.Printf(
					"[%s] [%v] [%d] %s %s %s",
					req.Method,
					time.Since(start),
					sw.statusCode,
					req.Host,
					req.URL.Path,
					req.URL.RawQuery,
				)
			}()

			next.ServeHTTP(sw, req)
		})
	}
}

Let’s see how the final logs look now:

Wow! Amazing, our custom logging middleware is complete now and looks good! Try playing around with it and you can print all cool things like user agent, host ip etc in the request logs according to your use case.

Please comment and share if you liked this post. If you have any suggestions or feedback, please comment, we will be happy to discuss and improve this post. Thanks a lot for reading!

Tagged : / / /

7 amazing tips for Go developer

I have just started learning Go and found it to be a very interesting language. It bridges the gap between rapid development and performance by offering high performance like C, C++ along with rapid development like Ruby, Python.

Through this blog, I wanted to share some behaviours of Go that i found tricky along with some style guidelines.

UN-EXPORTED FIELDS IN STRUCT CAN BE A MYSTERY

Yes it was a mystery for me when I started. My use case was simple, I was having an object of Person struct and wanted to marshal it using encoding/json package.

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    name string
    age  int
}

func main() {
    p := Person{name: "Anuj Verma", age: 25}
    b, err := json.Marshal(p)
    if err != nil {
        fmt.Printf("Error in marshalling: %v", err)
    }
    fmt.Println(string(b))
}

Output:

{}

Oh things worked fine without any error. But wait, why is the response empty ? I thought it must be some typo. I checked and checked and checked…
I had no idea why things were not working. Then I asked to every developer’s god(Google). You will not believe me but this was the first time  I understood the real importance of exported and un-exported identifiers in Go.
Since encoding/json is a package outside main and the fields inside our struct name and age are un-exported (i.e begins with a small case), therefore encoding/json package does not have access to Person struct fields and it cannot marshal it.

So to solve this problem, I renamed fields of Person struct to NameAge and it worked like a charm. Check here

JSON.DECODE VS JSON.UNMARSHAL ?

I was once writing an application which makes HTTP call to Github api. The api response was in JSON format.
So to receive the response and use it I created a Go struct (GithubResponse) matching the format of API response. The next step was to deserialise it. After looking up from the internet I came up with two possible ways to do it.

var response GithubResponse
err = json.NewDecoder(req.Body).Decode(&response)
var response GithubResponse
bodyBytes, _ := ioutil.ReadAll(req.Body)
err := json.Unmarshal(bodyBytes, &response)

Both will exactly do the same thing and will de-serialise a JSON payload to our go struct.  So I was confused about which one to use ? After some research I was surprised to know that using json.Decode  to de-serialise a JSON response is not a recommended way. It is not recommended because it is designed explicitly for JSON streams.

I have heard JSON, what is JSON stream ?
Example JSON:

{
  "total_count": 3,
  "items": [
    {
        "language": "ruby"
    },
    {
        "language": "go"
    }
  ]
}

Example JSON stream:

{"language": "ruby"}
{"language": "go"}
{"language": "c"}
{"language": "java"}

So JSON streams are just JSON objects concatenated. So if you have a use case where you are streaming structured data live from an API, then you should go for json.Decode. As it has the ability to de-serialise an input stream.
If you are working with single JSON object at a time (like our example json shown above), go for json.Unmarshal.

VAR DECLARATION VS :=

So this one is just a cosmetic suggestion, Remember when declaring a variable which does not needs an initial value prefer:

var list []string

over

list := []string{}

There’s no difference between them, except that the former may be used at package level (i.e. outside a function), but the latter may not. But still if you are inside a function where you have the choice to use both, It is a recommended style to use the former one.

Rule of thumb is to avoid using shorthand syntax if you are not initialising a variable.

IMPORTS USING BLANK IDENTIFIER

In one of my application we are using postgres database. I am using “lib/pq” which is a go postgres driver for database. I was going through the documentation here and I saw this:

import (
    "database/sql"

    _ "github.com/lib/pq"
)

func main() {
    connStr := "user=pqgotest dbname=pqgotest sslmode=verify-full"
    db, err := sql.Open("postgres", connStr)
    if err != nil {
        log.Fatal(err)
    }
}

Is this correct ? Why are we using an underscore in front of a package import. Checking on the internet I found that it is an anonymous import. It will import the package, but not give you access to the exported entities.

So the next question is very obvious:
If I do not have access to package entities, why we are importing it?

You remember when I said Go is an interesting language. In Go we can define an init() function in each source file, which allows us to setup things before the program executes. So sometimes we need to import a package so that its init() function gets called, without using the package directly in code.

Now lets understand why in code snippet above github.com/lib/pq is imported as a blank identifier. Package database/sql has a function

func Register(name string, driver driver.Driver)

which needs to be called to register a driver for database. If you have a look at this line from lib/pq library, things become more clearer. So lib/pq is calling the Register function to register an appropriate database driver even before our main function executes.

So even we are not using lib/pq directly from our code, but we need it to register driver postgres  before calling sql.Open().

USE SHORTER VARIABLE NAMES IN LIMITED SCOPE

In most of the languages you might have observed that it is advised to use descriptive variable names. For example use index instead of i. But in Go it is advised to use shorter variable names for variables with limited scopes.

For a method receiver, one or two letters is sufficient. Common variables such as loop indices and readers can be a single letter (ir). More unusual things and global variables need more descriptive names.

Rule of thumb is:

The further from its declaration that a name is used, the more descriptive the name must be.

Examples:

Good Style

// Global Variable: Use descriptive name as it can be used anywhere in file
var shapesMap map[string]interface{}

// Method
// c for receiver is fine because it has limited scope
// r for radius is also fine
func(c circle) Area(r float64) float64 {
  return math.Pi * r * r
}

EXPLICITLY IGNORE A JSON FIELD

If you want to ignore a field of struct while serialising/de-serialising a json, you can use json:"-". Have a look at an example below:

type Person struct {
    ID      int    `json:"-"`
    Name    string `json:"name"`
    Age     int    `json:"age"`
    Address string `json:"address"`
}

In above struct ID field will be ignored while serialising/de-serialising.

BACKQUOTES TO THE RESCUE

The back quotes are used to create raw string literals which can contain any type of character. So if you want to create a multi line string in Go, you can use back quotes. This will help you to save the effort for using escape characters inside string.

For example, suppose you want to define a string containing a JSON body:

{"name": "anuj verma", "age": 25}

See the below two ways:

b := "{\"name\": \"anuj verma\", \"age\": 25}"// Bad Style
b := `{"name": "anuj verma", "age": 25}`// Good Style

CONCLUSION

What am I missing here? Let me know in the comments and I’ll add it in. If you enjoyed this post, I’d be very grateful if you’d help it spread by sharing. Thank you.

Tagged : / /
%d bloggers like this: