What is a Go module?
What are a dependency and a dependency graph?
The Go approach to version selection.
How to use Semantic Versioning.
How to perform basic go Modules operations
\nDependency
Version
Semantic Versioning (SemVer)
Module
Modules have been introduced with version 1.11 of Go. Gophers can split their developments into separate code units that can be reused across other projects.
\n\nDevelopers reuse code that their peers developed to :
\nReduce the development time
\nSomebody might have already developed some basic functionalities team. Why taking time to develop it yourself?
What is your added value?
Reduce the maintenance cost.
The code that your team write needs to be maintained (ie. solving bugs, fixing vulnerabilities ...)
The code written by a dynamic community might be efficiently maintained.
\nAn important criterion when choosing a dependency is to check the community’s dynamism around the project.
A trendy project with many contributions might be “safer” than another maintained by a couple of developers.
A program may rely on ten other programs or libraries. The list of programs and libraries that a program uses is called its dependencies. We say that a program is dependent on another piece of software.
\nGo gives you the ability to use code written by others with Go modules easily.
\n\nA module is a group of packages (or a single package) that are (is) versioned. This group of go files forms together a module. Modules can be dependant on other modules.
\nA module is identified by a string which is called the “module path”.
The requirements (if any) of the module are listed in a specific file.
\nGo will use this file for each installation to build the module.
The file is named go.mod.
Go will also generate a file named go.sum. We will see later what it is.
The go.mod file has the following structure
\nmodule gitlab.com/maximilienandile/myAwesomeModule\n\ngo 1.15\n\nrequire (\n github.com/PuerkitoBio/goquery v1.6.1\n github.com/aws/aws-sdk-go v1.36.31\n)The first line gives the module path (it can be either a local path or a URI to a repository hosted on a version control system).
\nmodule is a reserved keyword. In the example, the module is hosted on gitlab.com on my account. The project is named myAwesomeModule.
The second line will give the version of Go used by the developer
Then another section define the dependencies that are used by the module :
require (\n DEPENDENCY_1_PATH VERSION_OF_DEPENDENCY_1\n DEPENDENCY_2_PATH VERSION_OF_DEPENDENCY_2\n //...\n)First, the module path and then the desired version :
\ngithub.com/PuerkitoBio/goquery v1.6.1We are loading the module github.com/PuerkitoBio/goquery hosted on GitHub And we require version v1.6.1.
\n\nThe go.mod file can be created automatically with the go command line :
\n$ go mod init my/module/pathThis will generate the following file :
\nmodule my/module/path\n\ngo 1.15For the moment, our go.mod file does not contain any dependency.
\nWe will list dependencies with the require keyword.
\nFor instance :
\nmodule gitlab.com/maximilienandile/myawesomemodule\n\nrequire (\n github.com/go-redis/redis/v8 v8.4.10\n)It means that the module gitlab.com/maximilienandile/myawesomemodule require the module github.com/go-redis/redis/v8 at version v8.4.10
\n\nYou can generate the go.mod file on an existing project with the go command line. Let’s take an example.
\nWe can create a file (main.go) that will import code from the repository gitlab.com/loir42/gomodule :
\npackage main\n\nimport "gitlab.com/loir42/gomodule"\n\nfunc main() {\n gomodule.WhatTimeIsIt()\n}The module we have imported has just one exposed function: WhatTimeIsIt (in its package timer) :
Here is the file content of timer.go :
\npackage gomodule\n\nimport "time"\n\nfunc WhatTimeIsIt() string {\n return time.Now().Format(time.RFC3339)\n}If you run the command go mod init a go.mod file will be created. It will not list your dependency yet:
\nmodule go_book/modules/appThen if you build your project by typing on your terminal go build the go.mod file will be modified, and the dependency is added to the require section :
\nmodule go_book/modules/app\n\nrequire gitlab.com/loir402/gomodule v0.0.1We did not specify anything for the required module version. Consequently, the build tool has retrieved the most recent tag from the remote repository.
\n\nIn the go.mod, you can explicitly exclude a version from your build :
\nexclude gitlab.com/loir402/bluesodium v2.0.1Note that you can exclude more than one module-version pair :
\nexclude (\n gitlab.com/loir402/bluesodium v2.0.1\n gitlab.com/loir402/bluesodium v2.0.0\n)It is possible to replace the code of a module with the code of another module with the directive “replace” :
\nreplace (\n gitlab.com/loir402/bluesodium v2.0.1 => gitlab.com/loir402/bluesodium2 v1.0.0\n gitlab.com/loir402/corge => ./corgeforked\n)The replaced version is at the left of the arrow; the replacement is at the right.
\nThe replacement module can be :
\nStored on a code sharing website (ex: Github, GitLab .…)
Stored locally
Some important notes :
\nThe replacement module should have the same module directive (the first line of the go.mod file).
Should the replacement specify a version? It depends on the location of the replacement :
\nDistant (Github, Gitlab) : required
Local: not required
A specific version of a module can be replaced OR all versions can be replaced.
\ngitlab.com/loir402/corge => ./corgeforkedwill replace all versions of gitlab.com/loir402/corge by a local version
\ngitlab.com/loir402/corge v0.1.0 => ./corgeforkedwill replace only version v0.1.0 by the local code.
API stands for Application Programming Interface. The most important letter is I. An API is an interface.
\nAn interface is like a frontier between two things, a “shared boundary across which two or more separate components of a computer system exchange information” (Wikipedia).
\nAn interface in computer science is a way for two different things to communicate. So what is an Application Programming Interface? It’s a set of constructs (constants, variables, functions ...) that are exposed to interact with a piece of software.
\nWith this definition we can say that the go package fmt exposes an API to the go programmer to interact with it. Its API represents the set of functions that you can use for instance Println. It also covers the set of exported identifiers of the package (constants, variables, types).
We can also say that the Linux kernel is exposing an API to interact with him, for instance, the function bitmap_find_free_region will tell the kernel to find a contiguous, aligned memory region.
=> Go Modules expose an API that is composed of all exported identifiers of the package(s) that the module is composed of.
\n\nPrograms naturally evolve:
\nAdditional functionalities are added (or removed)
Bugs are fixed (or created)
Performance is improved (or decreased)
Developers will add revisions to the original program, making it different (for the better or, the worse). Sometimes, a module’s API will evolve: for instance, a previously exported function is deleted.
\nPrograms relying on these particular functions will break (because it’s no longer exported).
\nDevelopers need to have a way to designate precise versions of a piece of code.\"1
\nA version is an unique identifier that designate a program at a specific revision and point in time.
\nThis unique identifier is generated based on a set of well-known and shared rules. The set of rules and the format of the identifier is called a “versioning scheme”. There are several schemes available. Go use Semantic Versioning. We will detail it in the next section.
\n\nTagged : it means that a “tag” has been created with a Version Control system.
A tag is a string (it’s name ) that designate a specific revision of a project maintained by a Version Control System.
\nEx : “v1.0.1” is a tag
The string “v1.0.1” is the tag name
It designates the code of a repository at a very specific revision.
Note that a tag can be any string.
\"GoIsMyFavLanguage\" is a valid tag.
A tag can designate a released version or a pre-released version of the software
\nA pre-release version (or release candidate) is considered to be ready. It is made available for last minutes tests.
Yet, it is not considered as a stable release (or just “release”).
Pre-release versions have specific tags with appended characters :
\nFor instance : 1.0.0-alpha, 1.0.0-alpha.1
Each project has different conventions about that
An untagged version is a specific state of the program at a given time.
\nIn the Git VCS, it’s a “commit”
The Git VCS will identify each commit with a SHA1 checksum (ex : 409d7a2e37605dc5838794cebace764ea4022388)
Semantic Versioning2 is a norm that Tom Preston-Werner wrote. Tom3. This specification defines the way version numbers are formed. It is widely used in the developer community.
\nIn this section, I will detail some important requirements of Semantic Versioning :
\n\nWhere :
\nthe major version
\nthe minor version
\nthe patch version (bug fix)
\nHow does it work? X, Y, and Z are positive numbers (without leading zeroes). Those numbers are incremented following a specific norm.
\nWhen you create new features that breaks the existing API of your software, you increment the major version number.
\nWhen you create new features or make performance improvement that do not break the existing API you increment the minor version number.
\nWhen you fix a bug in your code, you just increment the patch version
\nWhen you create a major version, you set to zero the minor and the patch version number. When you release a new feature, you set to zero the patch version.
\n\nDuring the beginning phase of a project, you know that things will change; your public API might not be the same later. During this phase of development, you can still create a version. The major number is then fixed to 0 but minor and patch version number can still be incremented. Other developers will know when they integrate your code into their software that your software is not considered as stable (meaning that the public API might change without notifications)
\n\nWhen you increment your major version number, you also communicate to others that you have developed changes that are not compatible with the previous major version.
\nLet’s take an example of an API change. A module called gomodule expose a function to the outside world called WhatTimeIsIt :
\npackage gomodule\n\nimport "time"\n\nfunc WhatTimeIsIt() string {\n return time.Now().Format(time.RFC3339)\n}The developer has released version 0.0.1 of this code. Later the same developer has received many complaints from others about the format of time. Fellow developers want to be able to specify a format for the time. RFC3339 is cool, but they want to choose. A new release is prepared :
\npackage gomodule\n\nimport "time"\n\nfunc WhatTimeIsIt(format string) string {\n return time.Now().Format(format)\n}The signature of the function has changed. As a consequence, code that uses this function will break if they import this new version. That’s why the developer has to create a new major version : 1.0.0.
\n\nOnce the version is created, we must not change the software. You cannot tag your software to a specific version, then delete the tag and recreate it with the same version number.
\n\nHandling dependencies is a complicated task, and many approaches exist. But first, I want to define the concept of a dependency graph.
\nThe dependency graph holds in a structured representation the dependencies and the sub-dependencies that are required for an application to work
The figure 1 reprensents the dependency graph of the myApp application. Each package is represented by a node (a circle). Nodes are also called vertices or leaves. For instance myApp is a node.
\nAn edge is a link between two nodes. Edges are represented with straight lines. In a dependency graph, edges have a direction represented by an arrow. Direct edges represent a dependency.
\nWe say that foo and bar are descendants of myApp.
\nThe package foo does not depend on anything. Foo has no descendant.
Bar depends on two other packages baz and qux.
Qux requires corge to work
Baz requires also corge to work
The dependency graph represents the dependencies required, but at the end, we need a list of dependencies to install our package. To create this final list, we need to respect the graph’s requirements. Dependency solving is the process of finding this final list.
\nIn our previous example, the final list will be :
\nFoo
Bar
Baz
Qux
Corge
Note that we only need to include Corge once, even if it is required two times (by Baz and Qux).
\n\nIn the previous example, we did not consider any version number. In this section, we will add version numbers.
\nIn the figure 2 we have represented the version numbers :
\nWe see that the node corge has been split into two new nodes in the graph. That’s because baz and qux depends on corge, but the two packages do not depend on exactly the same version.
\nWhich version should we add to our dependency list?
\nThe v2.0.0 or the version 2.7.0 ?
Both?
Just the more recent one, which is in this case v2.7.0?
We will see how Go handle this choice in the next section.
\n\nMinimal Version Selection (or MVS) is a set of algorithms4 used under the hood by the go command line to :
\nGenerate the go.mod file (and the go.sum), that lists all dependencies used by a project.
Update one or several dependencies
Downgrade one or several dependencies
MVS has been theorized by Russ Cox (one of the language developers). Russ wanted to create a system that was “Understandable. Predictable. Boring”
In this section, we will detail how it works by focusing on the main operations we will do every day:
\n\nEach module will give a list of module requirements
\nModules are identified by a module path
Each module required to specify a minimal compatible version
This list is in the go.mod file
Each module should follow the “import compatibility rule”.
\nWhen you add a new dependency to your project (by calling go get) it will :
\nDownload the module required
Add the module required to your go.mod file
Go will choose by order of preference
\nThe latest tagged stable release
The lastest tagged pre-release
The latest untagged version (latest known commit, also called latest pseudo-version)
The build list is the list of modules necessary to build a Go Program.
Each item of this list is composed of two things
\nA module path identifies a module
A revision identifier (which can be a tag or a commit id)
The steps required to create the build list for a given module are5 :
\nInitialize an empty list L
Take the list of modules required for the current module (go.mod)
For each module required
\nGet the list of modules required by this module (go.mod)
append those elements to the list L
repeat the operation for elements appended to the list
In the end, the list may contain multiple entries for the same module path.
\nTo display the final build list of a module, you can type the command :
\n$ go list -m allIn the following figure, you can see a mindmap that represents requirements for a module that requires only gitlab.com/loir402/bluesodium/v2 at version v2.1.1.
\nWe begin by listing the requirements of bluesodium at the requested version
It requires goquery
\nWhich requires itself cascadia
And net
\n, which requires crypto, sys, and text.
.…
In the end, we have the following list :
\ngithub.com/PuerkitoBio/goquery v1.6.1
github.com/andybalholm/cascadia v1.1.0
gitlab.com/loir402/bluesodium/v2 v2.1.1
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
golang.org/x/net v0.0.0-20200202094626-16171245cfb2
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
golang.org/x/text v0.3.0
golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01
The net module is required twice (in bold) :
\ngolang.org/x/net v0.0.0-20200202094626-16171245cfb2
golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01
The version golang.org/x/net v0.0.0-20200202094626-16171245cfb2 is prefered because it’s the most recent.
\n\nThe first time you add a dependency to your project, Go will download a specific revision :
\nA tagged version or,
A tagged prerelease or,
A specific commit
When you want to upgrade a dependency to its next version, you can type :
\ngo get -u gitlab.com/loir402/bluesodiumThe
\n-uflag will download the newer minor or patch6
\n\nYour project uses the version v1.0.1 of the module gitlab.com/loir402/bluesodium.
The maintainer released a new major version: tags beginning by v2 will appear on the repository :
\nA major version is released on a module you use in your program. When you attempt to upgrade it with the command :
\ngo get -u gitlab.com/loir402/bluesodiumIt will not download the newest major version (v2.0.1). Why?
\nVersion 2 of the module have a different path
This is the direct application of the “import compatibility” rule:
\nThe rule is: “Modules with the same module path should be backward compatible.”
Version 2 of a module introduces breaking changes. Those changes will impact the users of the previous versions.
Therefore, it should have a different module path
Let’s take a look at the go.mod file of the dependency :
\nmodule gitlab.com/loir402/bluesodium/v2\n\ngo 1.15The module path has changed; it’s no longer gitlab.com/loir402/bluesodium but it’s gitlab.com/loir402/bluesodium/v2.
When a module switch from v0 or v1 to v2, it should modify its path to comply with the import compatibility rule.
\nTo specifically require the major version 2, you need to launch the command. :
\ngo get -u gitlab.com/loir402/bluesodium/v2Because of the import compatibility rule, no breaking changes should be introduced (this operation only requests new patches and minor versions).
\nIf new versions are found then, the build list is modified AND also the go.mod file.
\nConcretely to do so, you will type the following command :
\n$ go get -u ./...\ngo: github.com/andybalholm/cascadia upgrade => v1.2.0\ngo: golang.org/x/net upgrade => v0.0.0-20210119194325-5f4716e94777The command will output the upgraded dependencies (here cascadia and net). Let’s take a look at the go.mod
\nmodule thisIsATest\n\ngo 1.15\n\nrequire (\n github.com/andybalholm/cascadia v1.2.0 // indirect\n gitlab.com/loir402/bluesodium/v2 v2.1.1\n golang.org/x/net v0.0.0-20210119194325-5f4716e94777 // indirect\n)Here you can see that the upgraded modules have been added to the go.mod and a comment “indirect” has been added. Why? Those lines are added to ensure that we use those specific upgraded versions when constructing the build list. Otherwise, the build list will stay the same.
\n\nIn this case, we want to update only one particular module to its newest version. The algorithm used is the following
\nA first build list is constructed as if no upgrades are made.
A second one is constructed with the requested upgrade.
Then the two lists are merged.
If a module is listed twice in the list, then the newest version is selected.
Downgrading a dependency is a common action. It might be necessary if one of the dependency used :
\nIntroduce a bug
Introduce a security vulnerability
Reduce the application performance
and many other reasons
In that case, the developer needs to be able to use a previous version of the dependency.
\nLet’s say that we used module E at version v1.1.0 and we want to downgrade the version of E to v1.0.0. Go will take each requirement of the application :
\nThe build list is constructed for that particular requirement
If the build list contains a forbidden version of E (ie. E at version v1.1.0 or above)
\nThen the version of that requirement is downgraded
If the downgraded version still contains E at version v1.1.0 (or above) it is downgraded at the previous version
The task is repeated until the forbidden versions are not in the build list.
This operation will potentially remove requirements that are no longer needed.
\n\nIn the previous section, we have seen that the go.mod file can exclude one module at a particular version (with the “exclude” directive).
\nIn that case, Go will find the next higher version (not excluded) of the module. It will search in the list of :
\nReleases
Prereleases
Note that pseudo-versions (commits) are excluded from the selection process.
\n\nThis is a set of characters that are generated by an algorithm (hashing algorithm) that takes as input data (strings, files ...).
One of the purposes of a checksum is to check the integrity of something quickly that was transmitted over a network.
\nThe transmission of data over a network is not 100% safe, and sometimes you can get a corrupted file (either due to the network or due to a hacking attempt).
If you compare the checksum that the author of the file generated with the one you generate yourself, you can be almost sure that you have the same file.
I say “almost” because the hashing algorithm you use can be too weak and generate identical checksum from different files. For instance, the MD5 algorithm can generate the same checksum for two very different files. We call this hash collisions.
The hash is the string of characters produced by the hashing algorithm. Hashing is different from encryption and encoding.
\nHash
\nA hashing algorithm outputs hashes of the same size.
It takes as input a piece of data (file, string, zip, ...)
Theoretically, you CANNOT go back to the hash’s original input.
Encode
\nThis is the process of converting a piece of data from a format to another format.
From the output, we CAN go back to the input.
Encrypt
\nThis is the process of taking an input, generally called the plaintext, and convert it to a ciphertext (the output).
The ciphertext is not understandable if you do not have the key.
To get the plaintext from the ciphertext, you need to be authorized.
To get authorized, you need to have the key.
The key can be symmetric or asymmetric
The go.sum file will contain cryptographic hashes of the module direct and indirect dependencies.7
\n\nLet’s first generate the go.mod file for the project myApp. In the terminal, we type :
\n$ cd myApp\n$ go mod initThe go.mod file generated is :
\nmodule gitlab.com/loir402/myAppIt’s empty! That’s because go mod init do not fill the go.mod file with the required dependencies. To avoid doing it manually, you can run the command
\n$ go installThis will update the go.mod file and create the go.sum file also :
\n// myApp/go.mod\nmodule gitlab.com/loir402/myApp\n\nrequire (\n gitlab.com/loir402/bar v1.0.0\n gitlab.com/loir402/foo v1.0.0\n)We have our two direct dependencies listed in the file: foo and bar. The tag that has been selected automatically is the most recent one: v1.0.0 for both dependencies.
\nGo install has also generated the following go.sum file :
\ngitlab.com/loir402/bar v1.0.0 h1:l8z9pDvQfdWLOG4HNaEPHdd1FMaceFfIUv7nucKDr/E=\ngitlab.com/loir402/bar v1.0.0/go.mod h1:i/AbOUnjwb8HUqxgi4yndsuKTdcZ/ztfO7lLbu5P/2E=\ngitlab.com/loir402/baz v1.0.0 h1:ptLfwX2qCoPihymPx027lWKNlbu/nwLDgLcfGybmC/c=\ngitlab.com/loir402/baz v1.0.0/go.mod h1:uUDHCXWc4HmQdep9P0glAYFdIEcenfXwuHmBfAMaEgA=\ngitlab.com/loir402/corge v1.0.0 h1:UrSyy1/ZAFz3280Blrrc37rx5TBLwNcJaXKhN358XO8=\ngitlab.com/loir402/corge v1.0.0/go.mod h1:xitAqlOH/wLiaSvVxYYkgqaQApnaionLWyrUAj6l2h4=\ngitlab.com/loir402/foo v1.0.0 h1:sIEfKULMonD3L9COiI2GyGN7SdzXLw0rbT5lcW60t84=\ngitlab.com/loir402/foo v1.0.0/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=\ngitlab.com/loir402/qux v1.0.0 h1:B1efJPpCgzevbS5THHliTj1owKfOi0Yo7tIaAm65n6w=\ngitlab.com/loir402/qux v1.0.0/go.mod h1:QexiArTQZcXRpFC3LLuGhk82aJoknf1n6c4WxlTeWdg=The go.sum file list all the project’s dependencies, the direct ones and the indirect ones. One dependency = two lines in this file. Let’s focus on the package foo :
\ngitlab.com/loir402/foo v1.0.0 h1:sIEfKULMonD3L9COiI2GyGN7SdzXLw0rbT5lcW60t84=\ngitlab.com/loir402/foo v1.0.0/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=The two lines have the following anatomy :
\nGo.sum will record the checksum of the module version used, but also the checksum of the go.mod file of the module; that’s why we have two lines per module.
\nThe hashing algorithm used is SHA256.
\nThe first checksum is the hash of all the files of the module.
The second checksum is the hash of the go.mod file of the module.
The hash is then converted to base64. The h1 string is fixed. It means that the Hash1 function inside of the go library is used8
To understand better how things work, I have created six projects hosted on GitLab :
\nhttps://gitlab.com/loir402/myApp
https://gitlab.com/loir402/foo
https://gitlab.com/loir402/bar
https://gitlab.com/loir402/baz
https://gitlab.com/loir402/qux
https://gitlab.com/loir402/corge
The first project myApp is our main project that has two direct dependencies : foo and bar. The other projects are indirect dependencies of myApp.
\nHere is the code of myApp :
\n// modules/example/main.go\npackage main\n\nimport (\n "fmt"\n\n "gitlab.com/loir402/bar"\n "gitlab.com/loir402/foo"\n)\n\nfunc main() {\n fmt.Println(foo.Foo())\n fmt.Println(bar.Bar())\n}In myApp we use the API of foo and bar. The package foo has no dependencies. Here is its code :
\n// foo/foo.go\npackage foo\n\nfunc Foo() string {\n return "Foo"\n}The package bar has two direct dependencies: baz and qux :
\n// bar/bar.go\npackage bar\n\nimport (\n "fmt"\n\n "gitlab.com/loir402/baz"\n "gitlab.com/loir402/qux"\n)\n\nfunc Bar() string {\n return fmt.Sprintf("Bar %s %s", baz.Baz(), qux.Qux())\n}And here is the package qux :
\n// qux/qux.go\npackage qux\n\nimport (\n "fmt"\n\n "gitlab.com/loir402/corge"\n)\n\nfunc Qux() string {\n return fmt.Sprintf("Qux %s", corge.Corge())\n}The package baz :
\n// baz/baz.go\npackage baz\n\nimport (\n "fmt"\n\n "gitlab.com/loir402/corge"\n)\n\nfunc Baz() string {\n return fmt.Sprintf("Baz %s", corge.Corge())\n}And finally our last package corge (which is a dependency of baz and qux) :
\n// corge/corge.go\npackage corge\n\nfunc Corge() string {\n return "Corge"\n}Note that this last one has no dependencies.
\nI have created for each one of those packages a version v1.0.0 on GitLab.
\n\nTo update all the dependencies of your project to the latest version (minor versions and patches) you can run the following command :
\n$ go get -u ./...Let’s test it!
\nWe will create a patch for the corge module. If you remember the previous section(about semantic versioning), a patch does not modify the module’s API (ie. everything that is exported by the module) :
\n// corge/corge.go\npackage corge\n\nfunc Corge() string {\n return fmt.Sprintf("Corge")\n}The signature is the same, but we just add a call to fmt.Sprintf.
On GitLab, I created a new tag to materialize that a new version is out! The tag is v1.0.1 (the last digit was incremented).
\nThen I will run the command go get -u inside the myApp folder. myApp has corge as an indirect dependency :
$ go get -u ./...\ngo: finding gitlab.com/loir402/corge v1.0.1\ngo: downloading gitlab.com/loir402/corge v1.0.1You see that Go has detected that we have a new patch for corge, and he has downloaded it. The go.sum file is now :
\n//...\ngitlab.com/loir402/corge v1.0.0 h1:UrSyy1/ZAFz3280Blrrc37rx5TBLwNcJaXKhN358XO8=\ngitlab.com/loir402/corge v1.0.0/go.mod h1:xitAqlOH/wLiaSvVxYYkgqaQApnaionLWyrUAj6l2h4=\ngitlab.com/loir402/corge v1.0.1 h1:F1IcYLNkWk/NiFtvOlFrgii2ixrTWg89QarFKWXPRrs=\ngitlab.com/loir402/corge v1.0.1/go.mod h1:xitAqlOH/wLiaSvVxYYkgqaQApnaionLWyrUAj6l2h4=\n//...and the go.mod file is :
\nmodule gitlab.com/loir402/myApp\n\nrequire (\n gitlab.com/loir402/bar v1.0.0\n gitlab.com/loir402/corge v1.0.1 // indirect\n gitlab.com/loir402/foo v1.0.0\n)Let’s note that :
\nThe previous version (v1.0.0) of corge is still into the go.sum file.
One line has been added to the go.mod file : “gitlab.com/loir402/corge v1.0.1”
The go.sum file keeps the old version for safety purpose because you might want to downgrade the dependency that you just upgraded (because of a bug for instance). In that case, you want to be sure to roll back to the same version that worked before the upgrade.
\n\nIf you do not want to update every dependency, you can target a specific one with the go get command.
\nFor instance, I have updated the code source of foo, and I have released a new version v1.0.1 (patch) :
\n// foo/foo.go\n// v1.0.1\npackage foo\n\nfunc Foo() string {\n return fmt.Sprintf("Foo")\n}Then we can run the following command into the terminal (inside the myApp directory) to update only “foo” to the latest version :
\n$ go get gitlab.com/loir402/foo\ngo: finding gitlab.com/loir402/foo v1.0.1\ngo: downloading gitlab.com/loir402/foo v1.0.1The go.mod file has been modified; foo is now required with the version v1.0.1 :
\nmodule gitlab.com/loir402/myApp\n\nrequire (\n gitlab.com/loir402/bar v1.0.0\n gitlab.com/loir402/corge v1.0.1 // indirect\n gitlab.com/loir402/foo v1.0.1\n)And the go.sum :
\ngitlab.com/loir402/foo v1.0.0 h1:sIEfKULMonD3L9COiI2GyGN7SdzXLw0rbT5lcW60t84=\ngitlab.com/loir402/foo v1.0.0/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=\ngitlab.com/loir402/foo v1.0.1 h1:6Dcvy69SCXzrGshVRDZzswqiA5Qm0n6Wt5VLOFtYF5o=\ngitlab.com/loir402/foo v1.0.1/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=The old version is still here (to downgrade safely), and the new version has been added.
\n\nTo target a specific version, you can also use the go get command :
\n$ go get module_path@XWhere X can be :
\nA commit hash
\nA version
\nGo will fetch the requested revision of the module and install it locally. Let’s take an example. I have made an evolution to the bar module :
\n// bar/bar.go\npackage bar\n\nimport (\n "fmt"\n\n "gitlab.com/loir402/baz"\n "gitlab.com/loir402/qux"\n)\n\nfunc Bar() string {\n return fmt.Sprintf("Bar %s %s", baz.Baz(), qux.Qux())\n}\n\nfunc Bar2() string {\n return fmt.Sprintf("Bar2 %s %s", baz.Baz(), qux.Qux())\n}I have added to the public API another exposed function Bar2. I have created a new minor version: v1.1.0.
\nLet’s update myApp to make it use this specific version :
\n$ go get gitlab.com/loir402/bar@v1.1.0\ngo: finding gitlab.com/loir402/bar v1.1.0\ngo: downloading gitlab.com/loir402/bar v1.1.0Let’s check what’s changed into the go.mod :
\nmodule gitlab.com/loir402/myApp\n\nrequire (\n gitlab.com/loir402/bar v1.1.0\n gitlab.com/loir402/corge v1.0.1 // indirect\n gitlab.com/loir402/foo v1.0.1\n)The version of bar has been updated to v1.1.0 into the go.mod. Let’s check what’s changed into the go.sum file :
\ngitlab.com/loir402/bar v1.0.0 h1:l8z9pDvQfdWLOG4HNaEPHdd1FMaceFfIUv7nucKDr/E=\ngitlab.com/loir402/bar v1.0.0/go.mod h1:i/AbOUnjwb8HUqxgi4yndsuKTdcZ/ztfO7lLbu5P/2E=\ngitlab.com/loir402/bar v1.1.0 h1:VntceKGOvGEiCGeyyaik5NwU+4APgyS86IZ5/hm6uEc=\ngitlab.com/loir402/bar v1.1.0/go.mod h1:i/AbOUnjwb8HUqxgi4yndsuKTdcZ/ztfO7lLbu5P/2E=The new version has been added to the list (the old version stays in the list)
\n\nIn some cases, a new version that has been published is not completely stable and occurs bugs in your application. In that case, you might want to downgrade it to the previous working version.
\nWe will simply run the same command as upgrading :
\n$ go get gitlab.com/loir402/bar@v1.0.0This will rollback the local version of bar to v1.0.0.
\n\nBefore releasing your app, you want your go.mod and go.sum file to reflect exactly what you use. For the example, we will use the module myApp. We will make it use a new dependency: grault. We will add it to the project then remove it. The objective is to see what happens to our go.sum and go.mod file.
\nThe source code of grault is quite the same as the other fake dependencies of our test. It exposes only a method Grault that send back the string “Grault”. Nothing new here :
\n// grault/grault.go\n// v1.0.0\npackage grault\n\nimport "fmt"\n\nfunc Grault() string {\n return fmt.Sprintf("Grault")\n}Let’s use it into myApp :
\n// myApp/main.go\npackage main\n\nimport (\n //...\n "gitlab.com/loir402/grault"\n)\n\nfunc main() {\n //...\n fmt.Println(grault.Grault())\n}When we run go install a new line has been added to the go.mod file and the go.sum file. Here is the go.mod :
\nmodule gitlab.com/loir402/myApp\n\nrequire (\n gitlab.com/loir402/bar v1.0.0\n gitlab.com/loir402/corge v1.0.1 // indirect\n gitlab.com/loir402/foo v1.0.1\n gitlab.com/loir402/grault v1.0.0\n)Now let’s remove the usage of grault and launch again go install. The go.mod file and the go.sum stay the same, but we no longer use grault. To clean up that we have to launch :
\n$ go mod tidy -v\n\nunused gitlab.com/loir402/graultThis will:
\nremove the mentions to grault in the go.mod and go.sum file
remove unused versions of dependencies inside the go.sum file
When you delete in go.sum the lines that refer to older versions of your dependencies, you might not get the same version of the module you used (when downgrading this module). When you downgrade, you want to be sure that the old version is the same as it was before. The cryptographic checksum stored into the go.sum file is here for that specific purpose.
\n\nThe different versions of the dependencies that have been downloaded by the go tool have been placed into the $GOPATH/pkg/mod directory. In the next listing, I have represented the tree of the folder mod :
\nFor gophers that have worked with other languages and their related dependency management system, the go.sum looks like a lock file. A lock file lists the dependencies that your project use along with a specific revision (a tag or a commit sha1).
\nHere is an example of a lock file for the Nodejs language (with versioning tool npm) :
\n{\n "name": "nodeLock",\n "version": "1.0.0",\n "lockfileVersion": 1,\n "requires": true,\n "dependencies": {\n "moment": {\n "version": "2.22.2",\n "resolved": "https://registry.npmjs.org/moment/-/moment-2.22.2.tgz",\n "integrity": "sha1-PCV/mDn8DpP/UxSWMiOeuQeD/2Y="\n }\n }\n}In this project, I used only one dependency called “moment” in version v2.22.2.
\nHere is another example of a lock file for a PHP project (using composer as dependency manager) :
\n{\n //...\n "packages": [\n {\n "name": "psr/log",\n "version": "1.0.2",\n "source": {\n "type": "git",\n "url": "https://github.com/php-fig/log.git",\n "reference": "4ebe3a8bf773a19edfe0a84b6585ba3d401b724d"\n },\n "dist": {\n "type": "zip",\n "url": "https://api.github.com/repos/php-fig/log/zipball/4ebe3a8bf773a19edfe0a84b6585ba3d401b724d",\n "reference": "4ebe3a8bf773a19edfe0a84b6585ba3d401b724d",\n "shasum": ""\n },\n "require": {\n "php": ">=5.3.0"\n },\n //....\n}How is it different from the go.sum file?
\nThe go.sum file stores the versions and the cryptographic sum of direct and indirect modules used by your application
\nThe aim of the go.sum file is to ensure that modules will not be altered at the next download.
Whereas the goal of a lock file is to allow reproducible builds.
Go use a deterministic approach to version selection.
\nThe build list will remain stable with time (if revisions used are not deleted by maintainer)
It means that the build list generated in January will not be different from the one generated in December.
Thus the introduction of a lock file is not necessary.
Dependency management systems that have introduced lock files generally don’t have such a deterministic approach
The lock file is needed to ensure that the builds of the application are reproducible by listing all the dependencies used and at which version.
You should commit your go.mod and go.sum in your VCS (for instance git), because :
\nOthers need the go.mod file to construct their build list.
Others will use the go.sum to ensure that the module downloaded have not been altered.
The go mod command has other interesting commands that you should know. This section will focus only on commands that I find interesting.
\n\ngo mod graph will output a dependency graph of your module to the standard output.
\nFor instance, here is the graph of myApp :
\ngitlab.com/loir402/myApp gitlab.com/loir402/bar@v1.0.0\ngitlab.com/loir402/myApp gitlab.com/loir402/corge@v1.0.1\ngitlab.com/loir402/myApp gitlab.com/loir402/foo@v1.0.1\ngitlab.com/loir402/bar@v1.0.0 gitlab.com/loir402/baz@v1.0.0\ngitlab.com/loir402/bar@v1.0.0 gitlab.com/loir402/qux@v1.0.0\ngitlab.com/loir402/qux@v1.0.0 gitlab.com/loir402/corge@v1.0.0\ngitlab.com/loir402/baz@v1.0.0 gitlab.com/loir402/corge@v1.0.0It’s not very visual, but it gives interesting information about myApp application. In the figure 4 I have represented it using arrows and circles. Each line of the output represents an edge between two packages. In this example, we have seven edges and six modules (foo, bar, baz, qux, corge, and myApp). Corge appears two times because myApp depends on corge v1.0.1, whereas qux and baz depend on corge v1.0.0.
\nThe command go mod vendor will :
\nHere is a tree view of the vendor folder of myApp :
\n# gitlab.com/loir402/bar v1.0.0\ngitlab.com/loir402/bar\n# gitlab.com/loir402/baz v1.0.0\ngitlab.com/loir402/baz\n# gitlab.com/loir402/corge v1.0.1\ngitlab.com/loir402/corge\n# gitlab.com/loir402/foo v1.0.1\ngitlab.com/loir402/foo\n# gitlab.com/loir402/qux v1.0.0\ngitlab.com/loir402/quxAs the name indicates it go mod verify will check your locally stored dependencies. Go will check that your locally stored dependencies have not been changed. This check is very useful to ensure that you are using the correct version of your dependencies and not an altered version. Those alterations can cause builds to fail.
\nThe command-line tool will check that for each module of the build list that the corresponding files downloaded located in pkg/mod/cache/download are not altered. Here is how the foder pkg/mod/cache/download is structured
\nYou can see that in this directory, Go has stored each version that we used for development. For each version, we have four different files :
\nVERSION.info
VERSION.mod
VERSION.ziphash
VERSION.zip
The info file contains the data at which it has been downloaded along with the version number :
\n{"Version":"v1.0.0","Time":"2018-11-03T19:36:07Z"}The .mod file is the exact reproduction of the original .mod file of the module. The file .ziphash contains the hash of the .zip file.
\nI have tried to modify the zipped version of the module and then to run go mod verify. Here is the error message that is displayed by go :
\ngitlab.com/loir402/foo v1.0.1: zip has been modified (/Users/maximilienandile/go/pkg/mod/cache/download/gitlab.com/loir402/foo/@v/v1.0.1.zip)A minor version introduces breaking changes. True or False?
What is the command to update a module to its latest version?
What is the name of the set of algorithms used in Go to manage dependencies?
Write a sentence describing a Module with the following words: packages, source files, go.mod, go.sum.
What is the purpose of the go.mod file?
What is the purpose of the go.sum file?
What is the command to initialize a module?
What is the command to display the build list of a module?
When the major version 2 is released, the module path is not modified. True or False?
A minor version introduces breaking changes. True or False?
\nFalse
Major versions introduce breaking changes
What is the command to update a module to its latest version?
\n$ go get -u path/of/the/moduleWhat is the name of the set of algorithms used in Go to manage dependencies?
\nCreate a sentence describing a Module with the following words: packages, go.mod, go.sum.
\nWhat is the purpose of the go.mod file?
\nThe go.mod file will define the module path of the current module
The go.mod file lists minimum versions of direct dependencies.
It also lists the minimum versions of indirect dependencies
\nEach dependency is identified by a module path.
It also gives the expected language version for the module
What is the purpose of the go.sum file?
\nWhat is the command to initialize a module?
\n$ go mod init path/of/the/moduleWhat is the command to display the build list of a module?
\n$ go list -m allWhen the major version 2 is released, the module path is not modified. True or False?
\nFalse
Because a new major version introduces breaking changes, the module path should change (to respect the import compatibility rule)
The string “v2” should be added to the module path
A Go module is a set of packages that are versioned together with a version control system (for instance, Git)
Go modules are identified by a module path.
\nA version is identified by a tag that describes the version changes.
To describe what changes are added in a version, we usually use a versioning scheme which is a set of rules enforced by developers
Semantic Versioning is a versioning scheme that Go uses
\nIn this scheme, a version number is a string formatted this way => “vX.Y.Z” (v is optional)
\nX, Y, Z are unsigned integers.
X is the major version number. When you increase this number, it means that you introduce breaking changes
Y is the minor number. This number should be increased when new non-breaking features are added.
Z is the patch number. This number is increased when a patch is created (a bug fix, for instance)
When a module publish a new major version greater or equal than 2, it should add a major version suffix to the module path
\nWe can initialize a Go module in an existing project by executing the go mod init my/new/module/path command.
To add a new direct dependency to a program, use the go get command:
\n$ go get my/new/module/to/add
To upgrade all your dependencies, use the go get command:
\n$ go get -u ./...
To upgrade one dependency, use the go get command:
\n$ go get -u the/module/path/you/want/to/upgrade
In the go.mod file, you can replace the code of a module by another one (stored on a code-sharing website or locally)
\nreplace gitlab.com/loir402/corge => ./corgeforkedYou can also exclude a specific version from your builds.
\nexclude gitlab.com/loir402/bluesodium v2.0.1https://en.wikipedia.org/wiki/Software_versioning↩︎
https://semver.org/↩︎
Tom is one of the cofounders of Github↩︎
In this section, we will see how MVS works. It is a transcription of the excellent Russ Cox’s article
Please note that the actual implementation of this algorithm is not as described. The actual implementation is based on graph traversal, which is more efficient than this approach. Take a look at the Russ Cox article to know more about that!↩︎
Source: go help get↩︎
https://golang.org/ref/mod section go.sum↩︎
if you are curious, here is the source code that generated the checksum: https://github.com/golang/go/blob/master/src/cmd/go/internal/dirhash/hash.go↩︎
Previous
\n\t\t\t\t\t\t\t\t\tInterfaces
\n\t\t\t\t\t\t\t\tNext
\n\t\t\t\t\t\t\t\t\tGo Module Proxies
\n\t\t\t\t\t\t\t\t