What is a package?
How are source files grouped?
Where to store the main.go file?
How to organize your Go project?
What is the import path? What are import declarations?
What is the go.mod file?
What is modular programming?
How to build modular applications with Go?
What is the internal directory, and why use it?
package
source file
import
module
modular programming
A Go program is a combination of packages (see figure 1).
\nA package is composed of one or more source files. Into those source files, the Go programmer declares :
\nconstant
variables
functions
types and methods
The package main is often composed of an individual file. The function main is the program entry point. In a Go program, you will also find a file named go.mod. The following sections will detail all those components.
\n\nIn the figure 2 you can see a schematic version of a source file. We will detail each part of the source file.
\nThe following snippet is an example of a source file that belongs to a packageoccupancy :
\n// package-imports/occupancy/occupancy.go\npackage occupancy\n\nconst highLimit = 70.0\nconst mediumLimit = 20.0\n\n// retrieve occupancyLevel from an occupancyRate\n// From 0% to 30% occupancy rate return Low\n// From 30% to 60% occupancy rate return Medium\n// From 60% to 100% occupancy rate return High\nfunc level(occupancyRate float32) string {\n if occupancyRate > highLimit {\n return "High"\n } else if occupancyRate > mediumLimit {\n return "Medium"\n } else {\n return "Low"\n }\n}\n\n// compute the hotel occupancy rate\n// return a percentage\n// ex : 14,43 => 14,43%\nfunc rate(roomsOccupied int, totalRooms int) float32 {\n return (float32(roomsOccupied) / float32(totalRooms)) * 100\n}At the top of the source file, we find the package clause in our example, it’s :
\npackage occupancyThe package clause is the first line of each source file. It defines the name of the current package.
\n\nThen the set of imports declarations.In this section of the source file, we define all the other packages that we want to use in this package. The package occupancy does not import other packages. Let’s take another example: here is a source file from the package room :
// package-imports/import-declaration/room/room.go\npackage room\n\nimport "fmt"\n\n// display information about a room\nfunc printDetails(roomNumber, size, nights int) {\n fmt.Println(roomNumber, ":", size, "people /", nights, " nights ")\n}\n\nimport "fmt"Here we import one package :
\nfmt wich is part of the standard libraryAfter the imports declarations, we find the most important part, the source code of the package. This is where you can declare variables, constants, functions, types, and methods.
\n\nWe must group the source files of a package into a single directory. The directory must have the same name as the package. For instance, the source files of the baz package must be stored into the baz folder (see figure 3 )
\nA Go program starts by initializing the main package and then run the function main from that package. The main package is where your program starts doing what it was built to do.
Here is an example of a main package :
\n// package-imports/main-package/main.go\npackage main\n\nimport "fmt"\n\nfunc init() {\n fmt.Println("launch initialization")\n}\n\nfunc main() {\n fmt.Println("launch the program !")\n}This program has an init function. This function can hold all the initialization tasks necessary for your program to run correctly (for more information about it, refers to the dedicated chapter).
The program also defines a main function. Both functions do not have a return type (unlike C, where the main function must return an integer).
The main function will be executed after all the initialization tasks have been performed. In this function, you usually call other packages and implement your program logic.
The previous program output :
\nlaunch initialization\nlaunch the program!The init function is launched, then the main function.
This is not always the case, but a project can have several main packages and thus several main functions. Usually, different main packages are present in large projects. Here are some common examples :
\nA main package for launching the web server of the application
Another main package to run scheduled database maintenance
Another one that has been developed for a specific punctual intervention...
For instance, the Kubernetes (one of the most starred Go projects) holds at the time of speaking 20 main packages.
\n\nNo, it’s not an obligation. You can do that if you have just one main function in your project. But often, it’s good practice to give it an informational name.
\nFor instance, in the project Kubernetes you can find the following files that are holding a main package :
\ncheck_cli_conventions.go
gen_kubectl_docs.go
We can infer what the program does just by looking at its name: the first one will check that CLI conventions are respected, the second one will generate docs.
\nPlease follow this convention. It allows other developers to understand what the program does by looking at the file tree.
\n\nAgain nothing is written in the Go specification about the folder that should contain the main package. There is still a strong usage main packages should live inside a cmd/ folder at the root of the repository.
\n\nHere is an example of a go.mod file :
\nmodule maximilien-andile.com/myProg\n\ngo 1.13The first line is composed of the word module followed by themodule path.
\nWe introduce here the notion of module. They have not always existed in Go; they have been introduced around March 2019. We will detail what modules are and how to use them in another section. For the moment, remember that a module is “a collection of Go packages stored in a file tree with a go.mod file at its root”1. When we add a package main to a module, we make it a program that can be compiled and executed.
\nWe need to define a module path for our module (our program). The path is the unique location of the module. Here are some examples of paths for two famous Go programs.
\nmodule github.com/hashicorp/vaultmodule k8s.io/kubernetes\"github.com/hashicorp/vault\" is an URL to a Github repository. A Github repository is a folder that contains the source code of a project. This folder can be publicly accessible (i.e., by anyone who has the link) or private. In the latter case, the code is only accessible to authorized developers. If you have access to a repository that has a go.mod inside, you can import it in your project. We will see later how that works.
Note that your program will not be automatically shared or exposed by Go if you choose a remote path! For testing purpose, you can choose a path that is not an URL :
\nmodule thisIsATest\n\ngo 1.13The next line of the go.mod file declares Go’s expected version. In this program, version 1.13 of go is expected. By setting it, we say to other developers that our project has been developed with this specific version of Go. In 10 years, Go will evolve, and some stuff that we use now in our program will probably not exists anymore. Programmers from the future will know how to compile it.
\n\nThe program is composed of 2 directories (in bold) :
\noccupancy
room
Each directory is composed of a go file. The file has the name of the directory. Each folder represents a package.
\n\npackage room\n\nconst roomText = "%d : %d people / %d nights"package occupancy\n\nconst highLimit = 70.0At the root directory of the program folder, we can find the go.mod file with the main.go file.
\n\nmodule thisIsATest\n\ngo 1.13package main\n\nimport "fmt"\n\nfunc main() {\n fmt.Println("program started")\n}Let’s try to build it and execute it :
\n$ go build main.go\n$ ./main\nprogram startedThe program does nothing except that it outputs the string “program started”. In each of the two packages, we defined a constant. Those two constants are never used.
\n\nAdd a package named booking to the previous Go program. In this package :
\nDefine a constant named vatRate with the value 20.0
Define a function named printVatRate that will print to the standard output the constant vatRate followed by the percentage sign.
A package is composed of a directory and at least one source file. If we want to create a package booking we should :
Create a directory named “booking”
Create a file into this directory. The name of the file will be booking.go
Here is the source file booking.go :
\n// package-imports/application-add-package/booking/booking.go\npackage booking\n\nimport "fmt"\n\nconst vatRate = 20.0\n\nfunc printVatRate() {\n fmt.Printf("%.2f %%", vatRate)\n}We can define a program in a single main.go file. This is what we did before. It’s legal; your code will compile. It’s legal, but it’s not the best solution. To understand why we need to take define the concept of “modular programming”.
\nThose questions are not new to software engineering and have been asked by many programmers before. During the sixties, software developers were struggling to maintain code bases. Softwares were designed into a single monolith with intensive usage of GOTO statements
The community needed a better way to develop. This was the beginning of the reflection on modular programming.
\n\nTo define a module, we will use Gauthier’s standard definition.
Performs a specific task
With inputs and outputs well defined
That can be tested independently
A large project that requires to perform several tasks can be split into different tasks, each one living inside a module with a well-defined API (2) and that can be tested independently from other systems.
\n\nWe can identify three main benefits
If you build your systems by using modules, you can make your team work on different modules independently, it increases the expected productivity of the team members. For instance, you are positioning two developers on module A and two others on module B. The two groups will not block each other if you have clearly defined the API of the two modules. Modules can also be implemented by others even if they are not finished yet.
\nModules increase the flexibility of development. You can ship isolated features into modules without having to do heavy changes on the existing code base of your system.
\nDevelopers can easily understand code organized in modules. A large system with a lot of structures, interfaces, and files is hard to understand. Joining a project that has thousands of files is hard. A system composed of modules requires less energy to be understood, modules can be studied iteratively. Modularity facilitates the system comprehension by developers (especially newcomers to the team)
\n\nDecomposition of a system into modules can be difficult. There is no unique answer to this issue. Here is some advice that is based on experience and my readings :
\nDo not create a module for every single task of a system; you will end up with too many modules with a size that is too small.
Instead, create modules that group functionalities that are close, for instance, a module for handling the database queries, a module to handle logging.
Inside your modules, the code is usually tightly coupled, meaning that the different parts of your module are strongly linked.
Between two modules, you should enforce loose coupling. Each module is treated as a component, and each component do not need another component to work. Modules should be independent. It allows you to replace one module with another one without touching the other modules.
We introduced three notions :
\nModular programming
Go packages
Go modules
Those three notions are linked together. Modular programming is a way of writing programs. The developer should create testable chunks of codes that perform specific tasks with a well-defined output and output. This “method” can be applied in C, Java, C++, Go...
\nGo packages are a tool that you can use to write modular programs. In a Go package :
\nWe can group source files with functions (or methods) related to a particular functionality :
\nEx: a package booking that will handle bookings for the hotel: create a booking, confirm a booking, cancel a booking...
Ex: a package room that will group functions related to hotel rooms: display the room information, check its current occupancy...
We can write tests that we can run independently of the rest of the code. (we will see how to do it in the unit test chapter)
Go modules are a way of grouping packages together to form an application or a library.
Finding a good name for a package is important. Other developers will use the package name in their source code, as a consequence, it must be informative but also shorts. A lot of blog articles have been written about this subject. In this section, I will give you some hints to choose your package name wisely :
\nThe name you choose must be small. In my opinion, no more than ten letters is a good length. Take an example from the standard library. Package names are very shot, often composed of a single word.
\nThe name has to bring essential information with it. A good test is to ask somebody what your package does without showing him the source code. A bad answer should make you think a second time about the name.
\nPackage names can be written in snake case (my_name) nothing in the specification is written about it, but the usage among the Go community is to use camel case (myPackageName)
\nSuch small names often disconcert beginners because they fear from potential name collisions. The risk of collision exists but not at the level of the package name :
\nAnother developer might have taken foo as a package name does not prevent me from choosing it because my package has not the same import path.
\nIn the previous chapter, all the code we produced was in the main package. We know how to create new packages that will contain functions, variables, constants... How to use functions from the package room into the package main?
\nHere is our package room :
\n// package-imports/trial-error/room/room.go\npackage room\n\nimport "fmt"\n\n// display information about a room\nfunc printDetails(roomNumber, size, nights int) {\n fmt.Println(roomNumber, ":", size, "people /", nights, " nights ")\n}How to call the function printDetails in the package main (our program’s entry point)? Let’s try to call the function directly :
// package-imports/trial-error/main.go\npackage main\n\nfunc main() {\n printDetails(112, 3, 2)\n}If we compile this program, we got this error :
\n./main.go:4:2: undefined: printDetailsThe function printDetails, which we try to call, is not defined in the package main. However, the function is defined in the package room. We want to import the package room into the package main. To allow the import to work, we need to say that our program is a Go module.
To do that, we need to create a go.mod file at the root of our project :
\nmodule thisIsATest2\n\ngo 1.13The module path is set to thisIsATest2. You can choose another one
Then we can import the room package into our main function :
\n// package-imports/trial-error-2/main.go\npackage main\n\nimport "thisIsATest2/room"\n\nfunc main() {\n printDetails(112, 3, 2)\n}Let’s try to build our program. We got an error message :
\n./main.go:3:8: imported and not used: "thisIsATest2/room"\n./main.go:6:2: undefined: printDetailsThis error says that :
\nWe import a package without using it.
The function printDetails is still not defined.
Go has no clue that we want to use the printDetails from the room package. To inform it, we can use this notation :
room.printDetails(112, 3, 2)It means : “call the function printDetails from package room”. Let’s try it :
// package-imports/trial-error-3/main.go\npackage main\n\nimport "thisIsATest2/room"\n\nfunc main() {\n room.printDetails(112, 3, 2)\n}If we compile, we got another error :
\n./main.go:6:2: cannot refer to unexported name room.printDetails\n./main.go:6:2: undefined: room.printDetailsIt gives us an indication: we are unable to call the function because it’s unexported... We need to export the function to make it visible to others! To do that in Go, you have to use a capital letter on the first letter of the function.
\nThis is the new version of the room package source file :
\n// room/room.go\npackage room\n\nimport "fmt"\n\n// display information about a room\nfunc PrintDetails(roomNumber, size, nights int) {\n fmt.Println(roomNumber, ":", size, "people /", nights, " nights ")\n}printDetails has been renamed to PrintDetails.
\nAnd this is our main function :
\n// package-imports/trial-error-4/main.go\npackage main\n\nimport "thisIsATest2/room"\n\nfunc main() {\n room.PrintDetails(112, 3, 2)\n}The function call has been changed to reflect the new name of the method (room.PrintDetails). Now we can launch the compilation of the program :
$ go build main.goAnd the execution :
\n$ ./main\n112 : 3 people / 2 nightsHooray ! it works. We have used an exported function from another package into the main package.
\n\nA function, a variable, and a constant if not exported is private to the package where it is defined
To export something just transform the first letter of its identifier to a capital letter
// package-imports/package-example/price/price.go\npackage price\n\n// this function is not exported\n// we cannot use it in another package\nfunc compute(){\n //\n}\n\n// this function is exported\n// it can be used in another package\nfunc Update(){\n //...\n}You might hear from other developers use the term public or private. Those terms are equivalent to exported / unexported. Other languages use them to refer to the same thing. This is not the official Go terminology.
\nExample usage: “Mmm, why did you make this function public, it’s not even used outside your package?”
Example usage: “Bob, you should consider making this method private”
The term “visibility” refers to the exported/unexported quality of a method.
\nTo import a package into another package, you need to know its import path. We will see how to determine it in the first section.
\nWhen you know the import path, you have to write an import declaration. This is the object of the next section.
\n\nThe Go specifications do not strictly specify the import path. The import path is a string that must uniquely identify a module among all the existing modules. The build system will use them to fetch the source code of your imported packages and build your program. Today the Go build system (which code live in the package go/build rely on different types of path :
\nStandard library path
\nThe source files that we will use to build your program are located by default in /usr/local/go/src (for Linux and Mac users) and in C:\\Go\\src for windows users
\nfmt, time ...URL that points to a code-sharing website.
\nGo has out-of-the-box support for the following code-sharing websites that use Git as a version control system (VCS): Github, Gitlab, Bitbucket.
The URL can be exposed on the internet (for instance, an open-source project hosted on Github)
\ngitlab.com/loir402/fooIt can also be only exposed in your local/company network
\nacme-corp.company.gitlab.com/loir402/fooLocal path or relative path
\nWhen downloading the package go will check the type of VCS system used in the HTTP response (inside a meta tag). If it’s not provided, you should add to the import path the type of VCS system used. Here is an example from Go official documentation:
\nexample.org/repo.git/foo/barVCS means Version Control System; it is a program that is designed to keep track of changes operated on source files of a project. VCS systems also have features that allow several developers to work on the same program simultaneously without annoying each other.
\nThere are several VCS programs available on the market :
\nGit
Mercurial
Bazaar
.…4
Git is Open source. It is very famous, and it is widely used today. (In my professional experience, I only used Git).
\nIt’s a program that you can use on the terminal, but also using a graphical interface.
\nThe changes to the source code of Go are managed using Git.
\nGit is a distributed VCS. Individual developers will fetch a copy of the source code by creating a “local repository”. When a developer is happy with its changes, he can push them to a remote repository.
\nGithub, Gitlab, Bitbucket : are websites that allow developers to host source code by initializing repositories hosted on their servers.
Those websites offer other features, but code hosting is their base activity.
Those websites are a great way to show and share your work with others. They are also the basic block of the open-source community. I strongly advise you to create a profile on that website to show what you can do and help others.
\n\nTo download a package, you will use the go get command inside your terminal.
For instance, if you want to import the code from https://github.com/graphql-go/graphql, open a terminal, and type :
\ngo get github.com/graphql-go/graphqlWe will cover this in details in the chapter dedicated to the dependency management system.
\n\nAn import declaration declares that we will use some external code in the current package.
\nAn import declaration can have different forms :
\n\nimport "gitlab.com/loir402/foo"Then in the file, you can access the exported element of the imported package with the syntax :
\nfoo.MyFunction()Here we use the function MyFunction from the foo package.
\n\nYou rarely have just one import. Often you have several packages to import :
\nimport (\n "fmt"\n "maximilien-andile.com/application2/foo"\n "maximilien-andile.com/application2/bar"\n)Here we import three packages. Here are their import paths :
\nfmt
maximilien-andile.com/application2/foo
maximilien-andile.com/application2/bar
You can also give your package an alias (a sort of surname for the package) to call its exported elements with this alias. For instance :
\nimport bar "gitlab.com/loir402/foo"Here we say that the package has the surname bar. You can use the exported types, functions, variables, and constants of the package foo (which import path is “gitlab.com/loir402”) with the qualifier bar like this :
\nbar.MyFunction()import ."gitlab.com/loir402/foo"With this syntax, all the functions, types, variables, and constants will be declared in the current package. As a consequence, all the exported identifiers of the package can be used without any qualifier. I do not recommend using this because it can become confusing.
\n\nWith a blank import, just the package’s init function will be called. A blank import can be specified with the following syntax :
\nimport _ "gitlab.com/loir402/foo"When you create a package and when your program is available on a code-sharing site (like Github, Gitlab ...), other developers will be able to use your package.
\nThis is a great responsibility for you. Other programs will rely on your code. It means that when you change a function or the name of an Exported identifier, other code might break.
\nTo forbid the import of packages, you can put them into a directory called “internal”.
\nIn the figure, you can see an example directory structure with an internal directory :
\ncmd/main.go contain the main package and the main function
internal/booking is the directory of the package booking.
internal/booking/booking.go is a source file of the package booking.
Any exported identifiers inside the booking package are accessible into the current program (ie. we can call a function from booking package into package main)
BUT, other developers will not be able to use it in their program.
One of your colleagues has created a program composed of a unique main.go file :
\n// package-imports/application-refactor/problem/main.go\npackage main\n\nimport "fmt"\n\nfunc main() {\n\n // first reservation\n customerName := "Doe"\n customerEmail := "john.doe@example.com"\n var nights uint = 12\n emailContents := getEmailContents("M", customerName, nights)\n sendEmail(emailContents, customerEmail)\n createAndSaveInvoice(customerName, nights, 145.32)\n}\n\n// send an email\nfunc sendEmail(contents string, to string) {\n // ...\n // ...\n}\n\n// prepare email template\nfunc getEmailContents(title string, name string, nights uint) string {\n text := "Dear %s %s,\\n your room reservation for %d night(s) is confirmed. Have a nice day !"\n return fmt.Sprintf(text,\n title,\n name,\n nights)\n}\n\n// create the invoice for the reservation\nfunc createAndSaveInvoice(name string, nights uint, price float32) {\n // ...\n}You are asked to refactor the code to improve its maintainability. Propose a new code organization :
\nWhich packages should you create?
Should you create new directories?
The initial task is to create a new directory and create a go.mod file. You can do that manually, but let me show you how to do it with your terminal :
\n$ mkdir application2Test\n$ go mod init maximilien-andile.com/packages/application2\ngo: creating new go.mod: module maximilien-andile.com/packages/application2This command will initialize automatically the go.mod file :
\nmodule maximilien-andile.com/packages/application2\n\ngo 1.13The module path is set to “maximilien-andile.com/packages/application2”. You can choose whatever you want.
\n\nThe main package defines three functions :
\nsendEmail
getEmailContents
createAndSaveInvoice
Two functions are related to emails. One function related to invoices. We can create one package for emails and one package for invoices. The rule is simple: group into packages constructs that are related to the same theme.
\nLet’s find names for our packages :
\ninvoice
The name has to remain short informative, and simple. Do not look for complexity; keep it simple and understandable. The next step is to create two directories to hold our source files.
\nInside the directory email, we will create a file named email.go :
\n// package-imports/application-refactor/solution/email/email.go\npackage email\n\n// send an email\nfunc Send(contents string, to string) {\n // ...\n // ...\n}\n\n// prepare email template\nfunc Contents(title string, name string, nights uint) string {\n // ...\n // TO IMPLEMENT\n return ""\n}Note that we changed the name of the functions.
\nsendEmail has been replaced with Send
getEmailContents has been replaced with Contents
The first thing to note is that the two functions are exported. Names have been modified. Because we are in package email, we do not need to call the method SendEmail. Here is two examples call :
\n// version 1\nemail.SendEmail("test","john.doe@test.com")\n// version 2\nemail.Send("test","john.doe@test.com")I hope that you prefer version 2... In version 1, we use the term email twice, which works but is not elegant.
\nWe also changed the name of the second function (getEmailContents). We changed two things:
\nIn other languages, methods that begin with the word “get” are pretty usual. In Go, it’s rare. Why? Because when you look at the function signature, the output is a string. You know that you will get your email contents as a string. Adding the “Get” do not bring more information to the caller. We get rid of it
We deleted the email reference because we are in the mail package. No need to repeat ourselves.
Here is the source code of the invoice package :
\n// package-imports/application-refactor/solution/invoice/invoice.go\npackage invoice\n\n// create the invoice for the reservation and\n// save it to database\nfunc Create(name string, nights uint, price float32) {\n // ...\n}The function name has been reworked: we export it, and we have removed the reference to invoice in the name. Here is an example of a call to the function Create :
\ninvoice.Create()If we did not change the name of the function, we could have something like that :
\ninvoice.CreateInvoice()This syntax repeats the word Invoice unnecessarily.
\n\n// package-imports/application-refactor/solution/main.go\npackage main\n\nimport (\n "maximilien-andile.com/packages/application2/email"\n "maximilien-andile.com/packages/application2/invoice"\n)\n\nfunc main() {\n // first reservation\n customerName := "Doe"\n customerEmail := "john.doe@example.com"\n var nights uint = 12\n emailContents := email.Contents("M", customerName, nights)\n email.Send(emailContents, customerEmail)\n invoice.Create(customerName, nights, 145.32)\n}First, note that we import two packages identified by their import path :
\nmaximilien-andile.com/package/application2/email
maximilien-andile.com/package/application2/invoice
The main function is the entry point of our program. Three variables are defined.
\ncustomerName (string)
customerEmail (string)
nights (unsigned integer)
Because we exposed the function Contents and Send, we are free to call them into the main package :
emailContents := email.Contents("M", customerName, nights)\nemail.Send(emailContents, customerEmail)Then we call the function Create (also exposed) :
invoice.Create(customerName, nights, 145.32)When you add this line :import _ \"github.com/go-sql-driver/mysql\" to the import section of your program, what will happen?
What is the syntax to import the package “github.com/PuerkitoBio/goquery” with the alias “goq” ?
How can you share your Go code with others?
How to spot exported identifiers in a package?
What the following line does:import _ \"github.com/go-sql-driver/mysql\" ?
This is a blank import declaration.
It is said to be blank because of the underscore character_.
All init functions of the package github.com/go-sql-driver/mysqlwill be run
What is the syntax to import a package with an alias?
\nimport goq \"github.com/PuerkitoBio/goquery\"How can you share your Go code with others?
\nCreate a git repository on a code hosting website (like Github, GitLab, bitbucket ...)
\nInitialize your module (withgo mod init gitlab.com/loir402/foo)
Push your code to the hosting website.
Share it with your colleagues and friends who will import it.
Note that you can also send your code by email or via physical mail, but it might be not optimal :)
How to spot exported identifiers in a package?
\nTheir first letter is capitalized
On the contrary unexported identifiers have not a first letter capitalized
\nex:const FontSize = 12 is an exported constant identifier
ex:const emailLengthLimit = 58 is an unexported constant identifier
A package is a group of source files that live in the same directory
A name identifies a package
Identifiers with a first letter uppercase are exported.
An exported identifier can be used in any other packages.
Packages that live inside an internal directory can be used inside packages of your module. However, they cannot be used by other modules.
A module is “a collection of Go packages stored in a file tree with a go.mod file at its root”6
https://blog.golang.org/using-go-modules↩︎
a secret management software (https://github.com/hashicorp/vault)↩︎
a system for managing containerized applications across multiple hosts (HTTPS://github.com/kubernetes/kubernetes)↩︎
You can see a list of available software here: https://en.wikipedia.org/wiki/List_of_version-control_software↩︎
Linus Torvald is the creator and principal developer of the Linux Kernel.↩︎
https://blog.golang.org/using-go-modules↩︎
Previous
\n\t\t\t\t\t\t\t\t\tFunctions
\n\t\t\t\t\t\t\t\tNext
\n\t\t\t\t\t\t\t\t\tPackage Initialization
\n\t\t\t\t\t\t\t\t