Learn Git The Hard Way

Ian Miell

This book is for sale at http://leanpub.com/learngitthehardway

This version was published on 2020-04-22

*   *   *   *   *

This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do.

*   *   *   *   *

© 2018 - 2020 Ian Miell

Table of Contents

1. Foreword

2. Learn Git the Hard Way

1. Introduction

3. Part I - Core Git

1. Introduction to Git

2. Git Basics

3. Cloning a Repository

4. Git Branching

5. Merging

6. Summary

4. Part II - Advanced Local Git

1. Git Stash

2. Git Add Interactive

3. Reflog

4. Cherry Picking

5. Git Rebase

6. Git Bisect

7. Summary

5. Part III - Remote Repository Management

1. Fetching and Pulling Content

2. Working With Multiple Repositories

3. Pushing Code

4. Git Submodules

5. Pull Requests

6. Summary

6. Part IV - Advanced Git

1. Git Log

2. Squashing Commits

3. Bare Repositories

4. Cherry-Picking and Three-Way Merges

5. Git Hooks

6. Summary

7. Finished!

Foreword

Git is such a ubiquitous technology in software now that it’s easy to take for granted. It’s also very easy to ‘get by’ with a little knowledge of it. Every day I come across very smart people who ‘only use three commands’ in git.

Usually that’s fine, but the reason I’ve come across them is because something’s happened and they’re bewildered, and they don’t know where to turn.

It’s my argument that the software community is woefully under-served when it comes to learning about git, and that mastering it pays massive dividends, and not just when using git!

If you want to learn git, you either find:

- Impenetrable man pages full of jargon that assumes you understand far more than you do

- One-liners to solve your particular problem, leaving you no better off the next time you want to do something

- Git ‘guides’ that are like extended man pages - theoretical, full of jargon, and quite hard to follow

All of the above is what this book tries to address.

If you’ve ever been confused by things like:

- Why your merge failed

- Why you can’t push your changes

- The difference between add and push

- What a ‘fast forward’ is

- What a ‘rebase’ is and why it’s important

then this book is for you.

It uses the ‘Hard Way’ method to ensure that you have to understand what’s needed to be understood to read those impenetrable man pages and take your understanding deeper when you need to.

Enjoy!

Learn Git the Hard Way

This git course has been written to help users to get to a deeper understanding and proficiency in git. It doesn’t aim to make you an expert immediately, but you will be more confident about using git and building your knowledge up from here.

Introduction

Why Learn Git The ‘Hard Way’?

The ‘Hard Way’ is a method that emphasises the process required to learn anything. You don’t learn to ride a bike by reading about it, and you don’t learn to cook by reading recipes. Books can help (this one hopefully does) but it’s up to you to do the work.

This book shows you the path in small digestible pieces based on my decades of experience and tells you to actually type out the code. This is as important as riding a bike is to learning to ride a bike. Without the brain and the body working together, the knowledge does not get there.

If you follow this course, you will get an understanding of git that can form the basis of mastery as you use it going forward.

What You Will Get

This course aims to give students:

- A hands-on, quick and practical understanding of git

- Enough information to understand what is going on as they go deeper into git

- A familiarity with advanced git usage

It does not:

- Give you a mastery of all git’s internals

- Give a complete theoretical understanding of all the subtleties and underpinning technologies of git

- Explain everything. There is plenty of time to go deeper and get all the nuances later if you need to

You are going to have to think sometimes to understand what is happening. This is the Hard Way, and it’s the only way to really learn. This course will save you time as you scratch your head later wondering what something means, or why that StackOverflow answer worked. You will also be able to construct your own solution and explain why that StackOverflow answer might not be perfect…

Sometimes the course will go into other areas closely associated with git, but not directly git-related, eg specific tools, terminal knowledge. Again, this is always oriented around my decades of experience using git and other source control tools, and I explain what’s needed as I go.

Assumptions

This book assumes some familiarity with very basic shell usage and commands. For those looking to get to that point, I recommend following this set of mini-tutorials:

https://learnpythonthehardway.org/book/appendixa.html

It also assumes you are equipped with a bash shell and a terminal. If you’re unsure whether you’re in bash, type:

echo $BASH_VERSION

into your terminal. If you get a bash version string output like this then you are in bash:

3.2.57(1)-release

If you are not in bash, then that’s not necessarily a problem, but be aware of it as a possible issue if you stumble (and let me know!).

I also assume you have a relatively up-to-date version of git. The git version on my Mac as I write this is:

git version 2.14.3 (Apple Git-98)

You can compare this to yours by typing this on the command line:

git version

Again, if you are not in the same version, then that’s not necessarily a problem, but be aware of it as a possible issue if you stumble (and let me know!).

How The Course Works

The course demands that you type out all the exercises to follow it.

Frequently, the output will not be shown in the text, or even described.

Any explanatory text will assume you typed it out. Again, this is the Hard Way, and we use it because it works.

This is really important: you must get used to working in git, and figuring out what’s going on by scratching your head and trying to work it out before I explain it to you. Eventually you will be on our own out there and will need to think for yourself.

Each section is self-contained, and must be followed in full to be sure that it makes sense. To help show you where you are, the shell command lines are numbered 1-n and the number is followed by a $ sign, eg:

1 $ first command

2 $ second command

At the end of each section is a set of ‘cleanup’ commands (where needed) if you want to use them.

Structure

This book is structured into four parts:

Part I - Core, Local Git

Core foundational concepts essential for managing git repositories locally.

Part II - Advanced Local Git

More advanced local repository management.

Part III - Remote Repository Management

Distributed management of git repositories.

Part IV - Advanced Git

This chapter delves a bit deeper into git for the user that wants to go further.

If you have any questions about the book before buying, please contact the author at:

ian@mail.meirionconsulting.com

Part I - Core Git

This part takes you through some fundamental concepts necessary to take your git knowledge further.

In it we cover:

- Source control as a concept

- Git vs traditional source control tools

- The four phases of the git lifecycle

- Benefits of git

- Git configuration

If you’ve used git before then you might think that these items are too simple. My experience of conducting this training with people - even users that have used git for years - is that most of them have an ‘a-ha!’ moment when going through these basics, so I would encourage you to go through it even if you feel you’re an intermediate user.

Introduction to Git

What is Source Control?

If you’re not familiar with source control, it solves a simple problem: how do you and your colleagues keep track of changes in your codebase?

You might start by:

- Sending each other updated files

- Emailing each other when files change

- Keeping tar files in a central location with version numbers

However, whenever the project scales in size, you encounter problems:

- If more developers are involved, there is a communication overhead to track all the changes

- If there are more projects running concurrently on the same codebase, then tracking what is changing what will get complicated

- If multiple developers are working on the same files at the same time, all the changes

- If there are more projects running concurrently on the same codebase, then tracking what is changing what will get complicated

- If multiple developers are working on the same files at the same time, something needs to co-ordinate the changes

A source control tool is a system that helps manage that complexity.

It’s a database of files and the histories of their states. Like a database, you have to learn the skills necessary to work on it before you feel the benefit.

I’m old enough to remember a time when people complained about using source control at all! These days, NOT using source control for projects is almost unheard of (but it still happens, believe me).

Before git existed, there were what might be called traditional source control tools.

Traditional source control tools (such as CVS and SVN) had a centralised architecture. You communicated with a server which maintained the state of the source code. This could mean several things:

- The source control database could get very big

- The history could get very messy

- Managing your checkouts of code could get complicated and painful

In the old world, if you checked out source code, that was a copy of some code that was inferior in status to the centralised version.

As far as the user was concerned, code was in one of two states:

- Local changes (“dirty”)

- Committed

Committed was synonymous with ‘pushed to the server’. My local changes could not be shared with anyone else until I committed and pushed them to the server.

Git, by contrast, is fundamentally distributed. Each git repository is a full copy of each other git repository it is copied from. It is not a “link” to a server, or a “shadow” copy of another repository. You can make reference to the origin repository, but you do not have to. All code databases (in Git, CVS or SVN) are known as repositories.

Now remember this, because I’ll be repeating it often:

ALL GIT REPOSITORIES ARE BORN EQUAL!

Git was created so people could work on the Linux kernel across the globe, and offline. So there is no concept of a central server that holds the ‘golden’ source. Instead people maintain their own source code database (ie their own repository) and reconcile, copy from, and integrate with other repositories.

Linus Torvalds (the creator of git and Linux) likes to joke that he’s made the Internet his back-up system.

The Four Phases of Git Content

In the git world you have four phases your code can go through:

The Four Phases of Git Content

Understanding these four stages are key to understanding git.

If this seems over-complicated now, it won’t as you grow to know and love git. If you’ve ever been confused by git, it’s likely because these stages were not understood properly.

You can get by with git without knowing too much about how it works, but you will hit limits in your understanding as you want to do more with it.

Don’t worry about memorizing the stages now, just be aware that it is important. Over time the understanding will become second nature.

Branches

In case you’ve not looked at a source control tool before, branching is a core concept.

A series of changes to a repository might look like this:

Three Commits to a Repository

Direction of Arrows in Git Diagrams

The arrows in the above diagram point         backwards in time. This is a convention standard to git diagrams to indicate that each commit depends on the previous commit back to the initial commit. In the above diagram, the first change (A) is depended on by B, which in turn is depended on by C.

The letters have no specific meaning here - they simply stand for a bunch of changes to lines grouped together.

Here change A is made, then B, then C. This might be informally called the main line. Most of the time in git it’s called the ‘master’.

But let’s say someone wants to make an experimental change but not affect the main line. Then they might branch the code at point C:

Code ‘Branched’ at Point C

Study the above diagram and make sure you understand it.

While the main line has continued with changes D and E, another branch has been created from C that has one change: F made to it. This experimental branch does not include the D and E changes, and the master branch does not include the experimental changes.

That way users can choose to get a view of the source on the main line branch or the experimental one.

That’s all a branch is: a set of changes from a specific point in time.

But What About GitHub?

Earlier I said that:

ALL GIT REPOSITORIES ARE BORN EQUAL!

In practice, some repositories are more equal than others (eg GitHub). This is a matter of convention within a project.

Most people use GitHub as their reference or upstream repository (ie the ‘primary’ one), but I could just as easily use a GitHub repository as a secondary or downstream repository for my workflow - it’s up to me (indeed I do this for some of my private repos).

Upstream? Downstream?

The phrases ‘upstream’ and ‘downstream’ are not always well-known (but many assume they are!). In coding terms, ‘upstream’ means the repository (or repositories) from which source code is taken, and ‘downstream’ means the repository (or repositories) that take changes to the code from the ‘upstream’. As with a stream of water, if you are ‘downstream’ you receive whatever floats along the water, and you have to ‘push’ changes ‘upstream’.

GitHub’s de facto status as a centralised repository (and all the machinery that assumes its existence and continuous uptime) is the reason every GitHub outage causes a flurry of smart-alec comments about git supposedly being a decentralised source control tool that relies on one central system.

More seriously, being a distributed source control tool makes Git more challenging to understand than traditional version control system (VCS) (aka source control management (SCM)) tools, which is one of the reasons why services like GitHub become central references: to keep things simple for the typical user.

Here’s a couple of diagrams that might help describe the differences between the decentralised and server-based VCS tools:

Centralised VCS Workflow

Distributed VCS Workflow

In a distributed source control, rather than everyone talking to one central source code ‘server’, the tool operates as a set of separate repositories that can ‘pull’ and ‘push’ changes to each other if the ‘owner’ of the repository agrees.

This is why the phrase ‘pull request’ exists. When you make a ‘pull request’ you are saying to the owner of the other repository ‘please take a change I’ve made in my repository and apply it to yours’. The operation could just as easily go the other way!

The first step to mastering git is understanding this equality of repos.

No GUIs here

In this book I focus on core git rather than GitHub, and the command line rather than GUIs (Graphical User Interfaces). This is for a few reasons.

One reason is that GUIs differ, and can mislead you about what is going on under the hood. This in turn can be confusing when you are forced to use (for example) BitBucket instead of GitHub.

Finally, it is easier to understand         core git and then map that to GUIs rather than the reverse.

The first step to mastering git is understanding this equality of repos.

Other Verson Control Systems (VCSes)

If you’re already familiar with other VCSes, git has some key differences you should bear in mind as you learn about it.

- History is more malleable.

You can change the history in your own copy of the repository and others’ (assuming you have the appropriate permission to push to them).

- Branching is cheap

In most traditional VCSes (such as CVS and Subversion) it’s very slow to branch a repository (O(n) (‘order n’) to number of files).

In git it’s an O(1) (an ‘order 1’ step).

Order Notation (eg O(1))

If you don’t know what that         O(1) notation means, here it just means that branching takes the same amount of time, regardless of the size of the repository you’re doing the branching in.

This makes experimentation with branching much easier.

Branch deletion is also a common and cheap operation.

This changes the typical workflow in a lot of cases.

- Commits are made across the whole project

In contrast to other source control tools, changes are made across the whole project, not per file.

One consequence of this is that moving/renaming files involves no loss of history for that file. This is a massive win over CVS, for example, which is file-oriented.

- No version numbers

Git does not automatically number versions of files/changes. It instead assigns a hash (effectively random) to the change to the repository which is used to refer to it.

Assumptions

At this point I assume you have:

- A command line to work with

- Access to basic Linux GNU tools (such as touch, grep)

- Installed git

Ensure that you have set your details up as per the below commands.

Run this to determine whether you are already good to go:

1 $ git config --list | grep ^user

If you saw two lines of output with user.email= and user.name= at the start, then you are already set up.

If you got nothing, then run the below commands, replacing with your email address and username:

1 $ git config --global user.email "you@example.com"

2 $ git config --global user.name "Your Name"

What You Learned

- What git is

- The four stages of git

- Setting up git

- Differences to other SC systems

- ALL GIT REPOSITORIES ARE BORN EQUAL!

Exercises

1) Install git and set up your config. Set up user.email and user.name using the --global flag.

2) Find out where the global git config is stored.

3) Research the other config items that are in the file and some of those that are not.

Git Basics

This section covers:

- git init

- The .git folder

- git log

- git status

- git add

- git commit

- git diff

This section is important because these are the basic tools you will most often use with git.

To initialise a git repository, run git init from within the root folder of the source you want to manage.

1 $ mkdir lgthw_git_basics

2 $ cd lgthw_git_basics

3 $ git init

This initialises a database in the folder .git locally. Your repository is entirely stored within this .git folder. There are no other files elsewhere on your filesystem you need to be concerned about to work with this repository. (There are config files for git, but these are global to the host. You can ignore them for now.)

Have a look at the files within your .git folder by typing in:

4 $ cd .git

5 $ ls

config

description

HEAD

hooks

info

objects

refs

It’s not part of the scope of this course to go into detail about all the git internals files seen here. You will cover some of them as we go into the course, though.

What is worth being aware of here are:

- The HEAD file

- The config file

HEAD

The HEAD file is key - it points to the current branch or commit ID you are currently ‘on’ within your git repository.

If you look at the file, you will see its contents contain the string:

refs/heads/master

This is an internal representation of the default master branch. Let’s have a look at the HEAD file.

6 $ cat HEAD

ref: refs/heads/master

The file’s contents links the ‘HEAD’ of the repository to the refs/heads/master file.

Git Configuration

The config file stores information about your repository’s local configuration, for example what branches and remote repositories your repository is aware of. Again, it’s a plain text file with a basic config structure:

7 $ cat config

[core]

repositoryformatversion = 0

filemode = true

bare = false

logallrefupdates = true

ignorecase = true

precomposeunicode = true

Your Output May Differ

Depending on how your git install was configured, the above output may differ. This is not a problem.

Again, don’t be concerned with understanding what it all means yet. Just be aware of its existence.

The git log Command

If you want to look at the history of this repository, run the git log command:

8 $ cd ..

9 $ git log

fatal: bad default revision 'HEAD'

You have a problem! This repository has no history to look at. So what happened?

Git has followed the HEAD pointer to the refs/heads/master entry and found nothing there! And indeed there is nothing there:

9 $ ls .git/refs/heads/master

ls: .git/refs/heads/master: No such file or directory

You will need to create a history for git log to return something useful.

The git status Command

As is often the case, git status is your friend:

10 $ git status

On branch master

Initial commit

nothing to commit (create/copy files and use "git add" to track)

Remember this command! A quick git status has got me out of many a sticky situation with git by telling me what is going on, and even advising me on what to do next.

Here it’s telling you where the HEAD is pointed at (the non-existent master branch), and that there is nothing to commit.

Create a file and check status again:

11 $ git status

12 $ touch mycode.py

13 $ git status

On branch master

Initial commit

Untracked files:

(use "git add <file>..." to include in what will be committed)

mycode.py

You are now advised that you have an untracked file. Git has detected that it exists but the repository is not aware of it.

Make git aware of it by adding it to the repository.

The git add Command

The add command tells git to start tracking files in what’s called the local index.

14 $ git add mycode.py

15 $ git status

On branch master

Initial commit

Changes to be committed:

(use "git rm --cached <file>..." to unstage)

new file:   mycode.py

You have added a file to the index ready to be committed to the repository.

Remember the four stages you looked at before:

The Four Phases of Git Content

You created your file ((1) local changes), then added/staged it to the index (2).

Still you have no history! Git has simply been made aware of the file, and you must make a commit to initiate git’s history.

15 $ git log

fatal: bad default revision 'HEAD'

So you will need to commit it to the repository to get a history.

The git commit Command

The git commit command tells git to take a snapshot of all added content at this point.

The EDITOR Variable

The below         git commit will bring up your shell’s configured editor (in the EDITOR environment variable) to save a file that contains the commit message for git to store. If you are confused at that point, you may want to look up shell EDITOR settings. cf http://askubuntu.com/questions/432524/how-do-i-find-and-set-my-editor-environment-variable

16 $ git commit

17 $ git log

commit e5fb099e952e8754b54f9b99be93d62e3fce0fca

Author: ianmiell <ian.miell@gmail.com>

Date:   Tue Apr 26 07:46:58 2016 +0100

Some message

Now that git is aware of the mycode.py file you can make a change to it and see how the local change looks using git diff.

The git diff Command

18 $ vi mycode.py

19 $ git diff

Again, you can see what’s going on by looking at the status. You can commit changes to files and add at the same time by doing git commit -a

19 $ git status

20 $ git commit -a

21 $ git status

git log now shows the history of the file:

22 $ git log

Cleanup

To clean up after the above work:

23 $ cd ..

24 $ rm -rf lgthw_git_basics

What You Learned

- git init

- The .git folder

- HEAD - a pointer to where in the history you are

- git log

- git status

- git add

- git commit

- git diff

Exercises

1) Create a git repository.

2) Add and commit a file to the repository.

3) Commit a few more changes, and then run git log to view the history.

Cloning a Repository

This section covers:

- git clone

- git reset

The git clone command is the way you create copies of git repositories to work on.

The git reset command is a way of returning to a previous or known state. As you play with git and learn it you will (and should!) often make mistakes in your local repositories. In these situations many users remove the entire repository and re-clone when often all that’s needed is a hard reset.

Cloning

In this section you’re going to play with the contents of your repository by deleting its content and seeing what your options are to recover state.

1 $ mkdir lgthw_git_clone

2 $ cd lgthw_git_clone

3 $ git clone https://github.com/ianmiell/shutit

4 $ cd shutit

5 $ ls .git

Cloning problems?

If you have problems cloning from GitHub, you can replace the clone with any URL that you can access from within your network. Otherwise, check your proxy settings.

There’s a .git folder, just as before. Remember that:

ALL GIT REPOSITORIES ARE BORN EQUAL!

This is a git repository just the same as the one you’ve cloned, and you own it. Its only connection with the repository you cloned from can be seen if you run:

6 $ grep -A2 'remote "origin"' .git/config

[remote "origin"]

url = git@github.com:ianmiell/shutit

fetch = +refs/heads/*:refs/remotes/origin/*

You will see a new section that indicates where this git repository was cloned from, and gives that remote a name by default: origin.

This is a sneak preview of what we will cover in Part III - Remote Repository Management. For now we are going to focus on learning about local repositories.

Accidental Deletion

Recall (again) the four stages of data in a git repositories:

The Four Phases of Git Content

Run these commands, in which you will look at the repository a little, and then make a disastrous mistake.

7 $ git log

This shows you a default history of the repository. Page through a few times by hitting ‘space’ or ‘down’ and you will see how far it goes back. Hit q to stop viewing it and return to the command line.

8 $ git log --oneline

Another way to view the log is one line per commit, which is much more concise and useful for many purposes. Obviously, some information is lost here.

9 $ git log --oneline --graph

Now you’ve added the --graph flag and you get a visual representation of the history. Parsing this graph can be tricky, so don’t worry about understanding exactly what you see yet. But keep it in mind if you ever have to figure out what went on in a repository’s past.

Using git log to view the history on the terminal is one of the most powerful tools you will have at your disposal if you are responsible for a shared repository.

Now you’re going to clone the repository so you’ve got a copy to mess around with.

10 $ cd ..

11 $ git clone shutit cloned_shutit

12 $ cd cloned_shutit

It’s worth at this point reminding yourself of the .git folder.

13 $ ls .git

Have a look around and see how it differs from the repository you originally cloned into the shutit folder.

Now you get distracted and type:

14 $ rm -rf ../cloned_shutit/*

Oops. You just deleted your cloned repository.

15 $ ls .git

Hmmm, why is the .git folder still there?

This is nothing to do with git, rather it’s due to how bash interprets the * character. Don’t worry about this if it makes no sense to you, but consider buying the sister book to this ‘Learn Bash the Hard Way’, available at https://leanpub.com/learnbashthehardway to learn more.

Just quickly here I will explain: the * character does not match files beginning with ‘.’ - the command ls ignores them similarly. This is just a convention of Linux (and other) operating systems.

You have cloned the repository, and ‘accidentally’ deleted all the files under git’s control in this cloned repository. This kind of disaster happens all the time, often when you start to use git, and more often when you are on a tight deadline.

What can you do?

One option often used is to wipe the entire checkout folder and re-clone, but there is another way, and it’s quite instructive.

The git reset Command

You can use git reset to recover the state of the git repository in various ways.

By default, git will recover whatever has been added to the index/staging area and place it in your working directory.

By contrast a, git reset --hard will blitz all local and added changes, reverting your checkout to a just-cloned and committed state.

Back to our friend git status:

16 $ git status

What does this tell you about the state you’re in?

Now you’re going to make another ‘mistake’:

17 $ git add .

Oh dear. You’ve not only deleted your files, you’ve added the deletions to the staging/index area of the git repository.

Help me, git status!

18 $ git status

In most versions, git status now reports that the deleted files are ready to be committed, but there’s also a helpful message at the top about using git reset (this message may vary depending on git version, or even not be there at all). If there’s a message like that, read it carefully, then type:

19 $ git reset --mixed

20 $ git status

Yes, I know that’s not what git status told you to do. But it’s the same thing --mixed is the default flag that git reset uses.

Based on what the git status output was, can you work out what point you’ve reverted to with git reset?

The --mixed flag takes items out of their ‘added’ status, but keeps them altered in the current working folder (or ‘working tree’, as the git often call it).

Now do the same delete/add cycle over, but git reset --hard this time rather than the default --mixed:

21 $ rm -rf ../cloned_shutit/*

22 $ git add .

23 $ git reset --hard

24 $ git status

What does git status tell you now? Have the files been returned? Have a look at what git log tells you.

So --hard not only takes items out of their ‘added’ status, they make the working tree state consistent with what was last committed. You can effectively lose your changes with the --hard flag.

What does this tell you about the .git folder? (Hint: how do you think the content was returned to the folder?)

What flag do you think should be the default to git reset? Why? (There’s no right answer, by the way, it’s just worth thinking about why the default was chosen, to embed the knowledge).

Cleanup

25 $ cd ../..

26 $ rm -rf lgthw_git_clone

What You Learned

- The git clone command

- Some new git log flags

- The git reset command

- The difference between git reset’s --hard and --mixed flags

Exercises

1) Check out a git repository from either your company repository or GitHub.

2) Browse the git log for that repository.

3) Look at the man page for git log and explore the options. Don’t worry about understanding everything in there, but play with the options and try to work out what is going on.

Git Branching

In this section you will learn about:

- git branch

- git checkout

We discussed branches conceptually in the first chapter of the book. If you’ve forgotten that, go back to the start and refresh your memory.

Now you’re going to see how branches are born in git.

Creating a Git Repository

Now you will create a git repository with a single file. This file will have separate changes made on two branches - master and newfeature.

Type these out to create a simple git repository:

1 $ mkdir lgthw_git_branch_1

2 $ cd lgthw_git_branch_1

3 $ git init

4 $ echo newfile > file1

5 $ git add file1

6 $ git commit -m 'new file1'

7 $ git status

Up to here you should be pretty comfortable, and know what’s going on. Now you’ll need to think:

8 $ git branch newfeature

9 $ git status

10 $ git branch

You’ve probably figured out that git branch newfeature created a branch called newfeature.

What branch are you on now? Was that what you expected?

Now add a commit on the master branch:

11 $ echo Line_master1 >> file1

12 $ git commit -am 'master change'

Now type these commands and see if you can work out the difference between them:

13 $ git log --decorate --graph --oneline

The attentive will have noticed that I slipped in a new flag there. The --decorate flag gives you useful information about the reference names (branches and tags) at various commit points. This is so handy it’s part of my muscle memory to add it to git log.

Is --decorate the default?

In recent versions of git, it is the default to show         --decorate if git is writing to a terminal even if it’s not supplied. If you output the git log command to a screen, and the redirect it to a file with the > operator, you might see a difference in content.

Now type this in and think about what’s going on:

15 $ git checkout newfeature

16 $ cat file1

17 $ echo Line_feature1 >> file1

18 $ git commit -am 'feature change'

19 $ git log --decorate --graph --oneline

20 $ git log --decorate --graph --oneline --all

Another flag (--all) was added there. As you’ve probably worked out, --all shows all branches, not just the branch you happen to be on. This is also in my muscle memory, but sometimes it produces insane graphs I can’t read, so I have to remember to remove it now and again.

Back to the code. You have added a ‘feature’ (ie a code change) to the newfeature branch. Now go back to the master branch and check that file1 doesn’t have the changes you made on the newfeature branch:

20 $ git checkout master

21 $ cat file1

This diagram shows the final state of the commit tree:

A Simple Branched Repository

This reflects the output of the last git log command. But note that it’s got a different orientation to the git log command! In the diagram the commits are shown left-right in time order, while git log shows them from top to bottom.

Take a moment to be sure you’ve understood that difference in representation.

Differently-oriented and confusing git diagrams are the norm in the git world, and we will come across this challenge again later.

git log Covered Later

The         git log command and its flags are covered in more depth in Part IV. This brief introduction is here to familiarize you with them as they are used in the course.

Note that the HEAD (and branch) moves forward with each commit.

The HEAD is where git is pointed at right now in your repository, and the branch is where that branch reference is pointed to.

Cleanup (Part One)

You need to type this in to clean up what you just did:

22 $ cd ..

23 $ rm -rf lgthw_git_branch_1

Detached Heads

Sometimes when using git you might have seen something like this:

$ git status

HEAD detached at 76d43b6

The idea of ‘detached heads’ sounds scary, and often is to people using git. But it needn’t be!

The HEAD pointer can be moved to an arbitrary point. In fact, git checkout does exactly this. You can specify a reference (like master or newfeature), or a specific commit id.

The next set of commands will check out the repository at a particular commit:

24 $ mkdir lgthw_git_branch_2

25 $ cd lgthw_git_branch_2

26 $ git clone https://github.com/ianmiell/shutit.git

27 $ cd shutit

28 $ git log

Hit q to exit the log output.

25 $ git checkout e36355ed00ac3af009d7113a9dd281c269a79afd

Note: checking out 'e36355ed00ac3af009d7113a9dd281c269a79afd'.

You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by performing another checkout.

[...]

Read the remaining text carefully and slowly, and make sure you’ve tried to understand what it all means.

Back to our best friend git status:

26 $ git status

HEAD detached at e36355e

nothing to commit, working directory clean

The detached HEAD part of this gives people the most concern. A detached HEAD sounds like a bad thing, and can be bewildering when you see it.

What it is is really quite simple, though. It just means that your git repository’s HEAD pointer (which is where git currently thinks it ‘is’) is not pointed at a branch at the moment.

Instead, it’s pointed at a commit ID, ie the HEAD is detached from a branch. There may well be a branch pointed at that commit (git log --decorate will tell you this), but your HEAD is not pointed at it.

Think about this, and ponder what advice you’d give to someone in a detached HEAD state to reorient themselves, given what you’ve been taught so far about branches and git log. Answering questions like this is the best way to understand git.

What About Tags?

We can cover off tags real quick while we’re here.

Tags are the same as branches except they have no history.

They point to a particular commit, and don’t change (unless you force a change, or delete it).

You can tag something where you are:

27 $ git tag i_was_here

or you can tag wherever a branch pointer is pointed at in your repository, even if you are not on it:

28 $ git checkout e36355ed00ac3af009d7113a9dd281c269a79afd

29 $ git branch -f newfeature

30 $ git checkout master

31 $ git tag remember_to_tell_bob_to_rewrite_this newfeature

Use git log to confirm that the tag now exists within the repository.

A Branch is Just a Pointer

Remember these points:

- A branch is a pointer to the end of a line of changes

- A tag is a pointer to a single change

- HEAD is where your git repository is right now

- ‘Detached head’ means you are at a commit that has no reference (branch or tag) associated with it

Is the master Branch Special

It’s worth also pointing out here that apart from its default status, there is nothing special about the master branch. It’s just a name. Your principal branch might be called live, alice, pristine, main, or whatever you like.

Cleanup (Part Two)

To clean up:

32 $ cd ..

33 $ rm -rf lgthw_git_branch_2

What You Learned

- The git branch command

- The git checkout command

- What ‘detached head’ means

- The --decorate flag to git log

Exercises

1) Clone a repository from GitHub and create a branch off the main branch (usually master) called mine.

2) Read up on git tag. Create a new commit on your branch and tag it.

3) Create another commit from there, and return to your previous commit by doing git checkout <COMMIT ID>. What does git status now say? Now get back to the mine branch.

4) Create a detached HEAD situation and make a commit. What do you expect to happen? Did it happen?

Merging

In this section you will learn about:

- git merge

- Merge conflicts and how to resolve them

- Merges and log histories

You’ve already covered basic branching in previous sections. As you will recall, branching gives you the ability to work on parallel streams of development in the same codebase.

In a sense, merging is the opposite of branching. When you merge, you take two separate points in your development tree and fuse them together.

It’s important to understand merging as it’s a routine job of a repository maintainer to merge branches together.

It’s also really important to take this chapter slowly and make sure you understand every step. It’s quite painful to get your head around merging properly, but once you do, your git power will rise significantly.

A Toy Merging Example

Now you’re going to run through a typical merging scenario by yourself. Take a look at this diagram:

A Branched Repository Pre-Merge

In this diagram, the repository is currently positioned at the tip of master (G). You know this because the HEAD is pointed at it.

If you merge the experimental branch into master with a git merge experimental command, you end up with a tree that looks like this:

A Branched Repository Post-Merge

A new change has been made (I). This change merges together the changes made on experimental with the changes made on master.

Merging, Step by Step

You can run through the above scenario step-by-step by following these commands:

1 $ mkdir -p lgthw_merging

2 $ cd lgthw_merging

3 $ git init

4 $ echo A > file1

5 $ git add file1

6 $ git commit -am 'A'

7 $ echo B >> file1

8 $ git commit -am 'B'

9 $ echo C >> file1

10 $ git commit -am 'C'

Now you are at this point of the process:

Step-By-Step Merge, Part One

Now branch to experimental and make your changes:

11 $ git branch experimental

12 $ git checkout experimental

13 $ git branch

14 $ echo E >> file1

15 $ git commit -am 'E'

16 $ echo H >> file1

17 $ git commit -am 'H'

and the repository is now in this state:

Step-By-Step Merge, Part Two

Return to master and make changes D, F and G:

18 $ git checkout master

19 $ git branch

20 $ echo D >> file1

21 $ git commit -am 'D'

22 $ echo F >> file1

23 $ git commit -am 'F'

24 $ echo G >> file1

25 $ git commit -am 'G'

Step-By-Step Merge, Part Three

and you are ready to merge:

26 $ git merge experimental

Auto-merging file1

CONFLICT (content): Merge conflict in file1

Automatic merge failed; fix conflicts and then commit the result.

Oh dear, that does not look good.

The merge failed with a CONFLICT message.

What’s Going On?

This is where a lot of people panic when merging, and it helps to understand what’s going on by running through it by hand with a toy example, as we are doing here.

So what exactly happens when you perform a merge?

When you run a merge, git looks at the branch you are on (here it is master), and the branch you are merging in (experimental), and works out what the first common ancestor is.

In this case, it’s point C, as that’s where you branched experimental.

It then takes the changes on the branch you are merging in from that first common ancestor and applies them to the branch you are on in one go.

These changes create a new commit, and the git log graph shows the branches joined back up.

Sometimes though, the changes made on the branches conflict with one another. That means the changes altered the same lines. In this case, the D, F and G of the master changed the same lines as the E and H of experimental.

Git doesn’t know what to do with these lines. Should it put the E and H in instead of the D, F and G, or put them all in?

If it should put them all in, then what order should they go in?

Changing lines around the same area in code can have disastrous effects, so by default git does not make a decision when this happens. Instead it tells you that there was a conflict, and asks you to ‘fix conflicts and then commit the result’.

If you look at file1 now:

A

B

C

<<<<<<< HEAD

D

F

G

=======

E

H

>>>>>>> experimental

all the lines from both branches are in the file.

There are three sections here:

1) The file up to line C is untouched, as there was no conflict.

2) Then we see a line with arrows indicating the start of a conflicting section, followed by the point in the repository that those changes were made on (in this case, HEAD).

3) Then a line of seven equals signs (=======) indicates the end of a conflicting set of changes, followed by the changes on the other, conflicting branch (the E and H on the experimental branch).

What you choose to do here is up to you as maintainers of this repository. You could add or remove lines as you wish until you were happy the merge has been completed, or replace it with something completely different. Git doesn’t care.

When you are satisfied you can commit your change, and the merge has been completed.

You could even leave the file as is (including the =======, and <<<<<<< lines), though this is unlikely to be what you want! It’s surprising how easily you can forget to resolve all the conflicting sections in your codebase when doing a merge.

When you are done you can commit the change, and view the history with the git log command.

27 $ git commit -am 'merged experimental in'

28 $ git log --all --oneline --graph --decorate

*   69441b0 (HEAD, master) merged

|\

| * b3d54fe (experimental) H

| * 4a013db E

* | d9d3722 G

* | bf0fc3e F

* | ccedaee D

|/

* 8835191 C

* f9e5b4f B

* 38471fe A

Reading this from bottom to top, you can read commit C and commit H as being merged into the HEAD of master.

Git Log Diagrams

git log prefers to show the history from most recent to oldest, which is the opposite of the diagrams in this book (which are left-to-right). The git man pages also like to show time from left to right, like this:

1                   A'--B'--C' topic

2                  /

3     D---E---F---G master

If you think this difference is confusing, I won’t disagree.

However, for git log it makes some sense: if you are looking at a repository with a long history, you are more likely to be interested in recent changes than older ones.

Cleanup

29 $ cd ..

30 $ rm -rf lgthw_merging

What You Learned

- What a merge is

- What a merge conflict is

- How to resolve a merge conflict

- How to read a merged log history

Exercises

1) Initialise a repository, commit a file, make changes on two branches and merge.

2) Read over git merge’s man page, and research what you don’t understand.

3) Create a merge knowing there will be a conflict and understand what you need to do to resolve.

4) Research merge tools that assist with merging issues so you are prepared if merging becomes something you do a lot.

Summary

In this section you have covered some of the basics of git. You learned about:

- The four stages of the git content lifecycle

- Ways git differs from other source control tools

- How git repositories are born, and that ALL GIT REPOSITORIES ARE BORN EQUAL!

- Various basic git commands, including add, commit, clone, branch, and checkout

- The .git folder and some of its contents

- what HEADs and ‘detached heads’ are

- What merging is, and what merge conflicts are

This is a lot of ground in a relatively short space of time, so make sure you have a grasp of all the above concepts. Don’t worry if you’re not expert or fully comforatble with them yet, but remember that if you stumble later it might be worth returning to some of these chapters and re-doing them. It’s much easier to learn git if the repository you are working on does not matter for your work.

In Part II you will cover some more advanced aspects of managing git repos locally before you tackle remote git repository management.

Part II - Advanced Local Git

Part I dealt with core git concepts, and setting up and managing code within a local git repository.

Part II deals with advanced local repository management and techniques, before moving onto to Part III, which looks at working with other git repositories.

In Part II you will cover:

- git stash

- git cherry-pick

- git rebase

- git bisect

Git Stash

Next you will learn a concept that you may end up using a lot!

Often when you are working you want to return to a pristine state, but not lose the work you have done so far.

Traditionally with other source control tools you’ve copied files that have changed locally aside, then updated your repository, and diffed and re-applied the changed files.

However, git has a concept of the ‘stash’ to store all local changes ready to reapply at will.

You can get very sophisticated with the stash, but 99% of the time I use it like this:

[do some work]

[get interupted]

$ git stash

[deal with interruption]

$ git stash pop

Here is a basic example of a change I want to ‘stash’:

1 $ mkdir lgthw_git_stash_1

2 $ cd lgthw_git_stash_1

3 $ git init

4 $ echo 'Some content' > file1

5 $ git add file1

6 $ git commit -am initial

7 $ echo 'Some changes I am not sure about' >> file1

Now let’s imagine I’m in the middle of some work, and Alice lets me know that there’s an important update to the code I need to pull from BitBucket.

First you can see what changes you have made locally with git diff:

8 $ git diff

diff --git a/file1 b/file1

index 0ee3895..5554e0f 100644

--- a/file1

+++ b/file1

@@ -1 +1,2 @@

Some content

+Some changes I am not sure about...

To store away these changes locally you run git stash:

9 $ git stash

Saved working directory and index state WIP on master: 34509a0 initial

HEAD is now at 34509a0 initial

A quick git status confirms that your working directory is ‘clean’:

10 $ git status

On branch master

nothing to commit, working directory clean

What happened to your change?

The really keen can look at:

11 $ git log --graph --all --decorate

*   commit 6a2fda32eaf55fedf90c3aa237a528cf7cf50a95 (refs/stash)

|\  Merge: 34509a0 9ff137c

| | Author: Ian Miell <ian.miell@gmail.com>

| | Date:   Tue Jun 28 12:02:45 2016 +0100

| |

| |     WIP on master: 34509a0 initial

| |

| * commit 9ff137cd51373afe6db37cbac4f1011b0db78ace

|/  Author: Ian Miell <ian.miell@gmail.com>

|   Date:   Tue Jun 28 12:02:45 2016 +0100

|

|       index on master: 34509a0 initial

|

* commit 34509a0afaf3eb9b7ff31dee3ab804903c8d36b0 (HEAD, master)

Author: Ian Miell <ian.miell@gmail.com>

Date:   Tue Jun 28 12:01:49 2016 +0100

initial

As you can see, it’s committed the state of the index (9ff…) and then committed the local change to the refs/stash branch, and merged them as a child of the HEAD on a new refs/stash branch.

Don’t worry too much about the details yet: it’s basically stored all the changes you’ve made (but not committed) ready to be re-applied.

The stash branch is a special one which is kept local to your repository. The ‘commit’ message WIP on master and index on master is added automatically for you.

The master branch is still where it was before you stashed, and the HEAD pointer is pointed at the master branch.

I can now do my other work (in this case, pulling the latest changes from a remote - remember, we cover remotes in Part III) without concern for whether it conflicts with those changes.

You can see your ‘stash list’ by running this command:

12 $ git stash list

stash@{0}: WIP on master: 34509a0 initial

Once ready, you can reapply those changes on the same codebase by running git stash pop:

13 $ git stash pop

On branch master

Changes not staged for commit:

(use "git add <file>..." to update what will be committed)

(use "git checkout -- <file>..." to discard changes in working directory)

modified:   file1

no changes added to commit (use "git add" and/or "git commit -a")

Dropped refs/stash@{0} (279ee87c68798caaf2ea3d45fcfa0ac42df6ba4b)

which ‘pops’ the zero-numbered change off the stash stack and restores the changes I stashed, applied to wherever I’ve ended up.

What’s a Stack?

A stack is a computer science concept of a series of items stored in such a way that you can ‘push’ an item to the top of it any number of times, and then ‘pop’ an item off the top until it’s empty. When you pop, you only get the most recent item that was ‘pushed’ on. This means that when you ‘pop’ your git stash, you get the most recent one you ‘pushed’ (which is likely to be what you wanted).

Now you can cleanup the previously-created folder:

14 $ cd ..

15 $ rm -rf lgthw_git_stash_1

Choosing Your Stash

You may be wondering at this point how you manage multiple stashes that are on the stack.

Type this sequence out. It will stash two similar-looking changes.

17 $ rm -rf lgthw_git_stash_2

18 $ mkdir lgthw_git_stash_2

19 $ cd lgthw_git_stash_2

20 $ git init

21 $ echo 'Some content' > file1

22 $ git add file1

23 $ git commit -am initial

24 $ echo 'First changes I am not sure about' >> file1

25 $ git stash

26 $ echo 'Second change I am also not sure about' >> file1

27 $ git stash

28 $ git stash list

stash@{0}: WIP on master: d3f21d2 initial

stash@{1}: WIP on master: d3f21d2 initial

You can see you now have two changes in your stash’s stack. But which is which? Is stash@{0} the first change, or the second one?

Some minimal information is available if you type git stash show <ID>:

29 $ git stash show stash@{0}

file1 | 1 +

1 file changed, 1 insertion(+)

30 $ git stash show stash@{1}

file1 | 1 +

1 file changed, 1 insertion(+)

but this is not sufficient for you to tell what is going on. It just tells you that one line was inserted in each change.

git stash show --patch <ID> gives you information in diff format also:

31 $ git stash show --patch stash@{0}

diff --git a/file1 b/file1

index 0ee3895..c8f5c78 100644

--- a/file1

+++ b/file1

@@ -1 +1,2 @@

Some content

+Second change I am also not sure about

32 $ git stash show --patch stash@{1}

diff --git a/file1 b/file1

index 0ee3895..aa51db4 100644

--- a/file1

+++ b/file1

@@ -1 +1,2 @@

Some content

+First changes I am not sure about

From this you can infer that stash pushes to a stack at number zero, and then pops from zero if you use git stash pop.

If you want to apply the first change only from here, run:

33 $ git stash apply stash@{1}

On branch master

Changes not staged for commit:

(use "git add <file>..." to update what will be committed)

(use "git checkout -- <file>..." to discard changes in working directory)

modified:   file1

no changes added to commit (use "git add" and/or "git commit -a")

34 $ git diff

diff --git a/file1 b/file1

index 0ee3895..aa51db4 100644

--- a/file1

+++ b/file1

@@ -1 +1,2 @@

Some content

+First changes I am not sure about

Be aware of a little gotcha here - if you apply a git stash, then it remains in the list. git stash pop will then remove the stash item for you.

35 $ git stash list

stash@{0}: WIP on master: d3f21d2 initial

stash@{1}: WIP on master: d3f21d2 initial

Go ahead and remove the stash items yourself.

Cleanup

To clean up:

36 $ cd ..

37 $ rm -rf lgthw_git_branch_2

What You Learned

- What the stash is

- How the stash stack works

- How to re-apply changes

Exercises

1) Stash several changes and then re-apply them in a different order, ending up with an empty stash list.

2) Find out how to manually remove a stash entry.

Git Add Interactive

This section covers:

- Interactively adding in git

- Splitting hunks

- The difference between staging and committing

Recap

Previously you’ve learned about the four stages of working in Git:

The Four Phases of Git Content

So far you’ve shown a difference between adding (staging) and committing, but this still causes confusion for people - what’s the point of this?

git add -i

Let’s demonstrate how you might want to use this with a simple example:

1 $ mkdir lgthw_add_i

2 $ cd lgthw_add_i

3 $ git init

4 $ echo 'This is file1' > file1

5 $ echo 'This is file2' > file2

6 $ git add file1 file2

7 $ git commit -am 'files added'

8 $ cat > file1 << END

9 > Good change

10 > This is file1

11 > Experimental change

12 > END

13 > $ cat > file2 << END

14 > All good

15 > This is file2

16 > END

17 $ git add -i

Now run the following, and input the characters (or just ‘return’) at the appropriate points:

25 $ git add -i

staged     unstaged path

1:    unchanged        +2/-0 file1

*** Commands ***

1: status       2: update       3: revert       4: add untracked

5: patch        6: diff         7: quit         8: help

19 What now> p

staged     unstaged path

1:    unchanged        +2/-0 file1

20 Patch update>> 1

staged     unstaged path

* 1:    unchanged        +2/-0 file1

Note that in the next line, you only need to input the ‘return’ key. Just hit return.

21 Patch update>>

diff --git a/file1 b/file1

index 6a00e12..014f6e4 100644

--- a/file1

+++ b/file1

@@ -1 +1,3 @@

+Good change

This is file 1

+Experimental change

22 Stage this hunk [y,n,q,a,d,/,s,e,?]? s

Split into 2 hunks.

@@ -1 +1,2 @@

+Good change

This is file 1

23 Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]? y

@@ -1 +2,2 @@

This is file 1

+Experimental change

24 Stage this hunk [y,n,q,a,d,/,K,g,e,?]? n

*** Commands ***

1: status       2: update       3: revert       4: add untracked

5: patch        6: diff         7: quit         8: help

25 What now> q

Bye.

26 $ git status

27 $ git diff

Now you have staged the good change, but not lost the other changes you have made. This gives you more granular control over the changes committed.

Hunks?

A lot went on in that last block of output so it’s worth reading over the above carefully.

First you were presented with a set of choices:

1: status       2: update       3: revert       4: add untracked

5: patch        6: diff         7: quit         8: help

We’re only going to focus on ‘patch’ (number 5) here. There’s no point exhaustively listing all the choices and their meanings, as most are self-explanatory and the remainder I have never needed to use, but if you’re interested there’s an exercise below to go and figure it out.

As you have probably figured out already, a ‘hunk’ is a contiguous (or nearly contiguous) section of a diff.

You’re then presented with a set of numbered changes. Above there was only one, but you might have had multiple.

staged     unstaged path

1:   unchanged        +2/-0 file1

The first ‘staged’ column tells you what has been staged so far. The second ‘unstaged’ column tells you how many lines have been added/removed. In the above example, two lines have been added, and none removed. The third is the ‘path’ of the file.

An asterisk (*) indicates that the option is the chosen one, so by hitting that number, followed by the ‘return’ key, you only need to further hit the ‘return’ key once to choose the first ‘hunk’. Then you are presented with the hunk itself, and a bewildering series of options:

diff --git a/file1 b/file1

index 6a00e12..014f6e4 100644

--- a/file1

+++ b/file1

@@ -1 +1,3 @@

+Good change

This is file 1

+Experimental change

Stage this hunk [y,n,q,a,d,/,s,e,?]?

At this point you chose ‘s’, which stands for ‘split’ the hunk.

If you’re not sure what’s going on, you can choose the ‘?’ option here, which explains what the various options mean. If I’m stuck I can never remember what they all do, so I depend heavily on ‘?’. Read through the options now and make sure you understand them all.

Once split, you can ‘stage’ the hunks one at a time by choosing ‘y’ for ‘yes’ when prompted.

If you are happy with the changes you can go ahead and commit all the changes you have made.

Why Split Hunks?

Why is this splitting useful? It’s most commonly used to avoid committing lines you might want to keep for local development, but not persist in the history of the repository.

A common example of this is printed debug lines that you don’t want getting to production, but do want to keep for local development. Another example might be chunks of notes that only make sense to you while you are developing.

Why Stage at All?

Committing will commit all the changes you have staged. What is the point of staging then?

It is to confirm that you want to commit some changes made locally, but not others.

These changes are added to the index (as opposed to the repository).

Remember: index == staging == adding

Committing commits your staged changes to the local repository (as opposed to adding), after which these commits can be pushed to remote repositories.

If (like me) you run git commit -am "your commit message" frequently, you skip over these steps, which can result in commits with stray lines that you don’t want to be part of the history.

The -a flag stands for ‘automatically add’. Confusingly the -a flag is aliased to --all, even though not all files are necessarily added - only already-added ones are.

Cleanup

To clean up:

28 $ cd ..

29 $ rm -rf lgthw_add_i

What You Learned

- The difference between staging and committing

- Why the distinction exists

- How to stage specific hunks of code to the index

Exercises

1) Research what the other interactive options do.

2) Research what the other hunk staging options do.

3) Research commit --interactive and learn to use it. Why not use git commit -i?

Reflog

This section covers:

- The git reflog command

- A revisit of the git reset command

In this section we’re going to look at the reflog.

The reflog gives you references to a sequential history of what you have done to the repository. This can come in very handy when you play with your local repository’s history, as you will see here.

Lose a Commit, Get it Back

First set up a repository with two commits:

1 $ mkdir lgthw_reflog

2 $ cd lgthw_reflog

3 $ git init

4 $ echo first commit > file1

5 $ git add file1

6 $ git commit -m file1

7 $ echo second commit >> file1

8 $ git commit -am 'commit message for file1.1'

9 $ git log

Then do some magic to effectively remove the last commit:

11 $ git checkout HEAD^

12 $ git branch -f master

13 $ git checkout master

14 $ git log

What was that?

Don’t worry about what you just did - it’s a more advanced set of commands that mess with git’s history. We’ll cover it later.

The last commit has disappeared! You have fully reverted the master branch to where it was before. Even git log --all does not show it, because it’s not on a branch.

Don’t worry about the details of what you did, the point here is to create a situation in git that you want to get out of.

This is where git’s reflog can help.

Git reflog records all movements of branches in the repository. As with git stashes, it is local to your repository.

16 $ git reflog

66cdcd2 HEAD@{0}: checkout: moving from 66cdcd23c5c005edecd7cd7b162d7b42b7a02ab4 to \

master

66cdcd2 HEAD@{1}: checkout: moving from master to HEAD^

40e99f7 HEAD@{2}: commit: commit message for file1.1

66cdcd2 HEAD@{3}: commit (initial): file1

Reflog?

The reflog is called that because it’s a ‘REFerence LOG’. I always think of someone being flogged, but that’s probably memories of painful git experiences…

Git’s reflog is a history of the changes made to the HEAD (remember the HEAD is a pointer to the current location of the repository).

Use git reset to Restore State

If you git reset --hard the repository to the reference given (in this case 40e99f7 - your ID will differ!):

16 $ git reset --hard 40e99f7

HEAD is now at 40e99f7 commit message for file1.1

17 $ git log

you are returned to where you were.

The --hard flag updates both the index (staging/added) and the working tree, as you saw previously.

The reflog contains refernces to the state of the repository at various points even if those points are no longer apparently reachable within the repository.

Cleanup

To clean up:

18 $ cd ..

19 $ rm -rf lgthw_reflog

What You Learned

- What git’s reflog is.

- The git reflog command.

- Another use for the git reset --hard command.

Exercises

1) Do the above section again, but this time create another branch from master after the second commit. What changes this time as you run through the rest. Why?

Cherry Picking

This section covers:

- git cherry-pick

- A revisit of git log

- A revisit of git branching

This section is important because cherry-picking is the simplest-to-understand means to move commits around. This is the basis of more sophisticated movement of commits such as rebasing and ‘squashing’ commits.

A Simple Cherry-Pick

Since every commit in git is a change set with a reference id, you can easily port changes from one branch to another.

To demonstrate this, create a simple repository with two changes:

1 $ mkdir lgthw_cherry_pick

2 $ cd lgthw_cherry_pick

3 $ git init

4 $ echo change1 > file1

5 $ git add file1

6 $ git commit -am change1

7 $ echo change2 >> file1

8 $ git commit -am change2

9 $ git log --all --oneline --decorate --graph

At this point you branch off into two branches, master and experimental.

10 $ git branch experimental

11 $ git checkout experimental

Now insert the following line before the first line in file1 with an editor of your choice:

crazy change

12 $ cat file1

13 $ git commit -am crazy

14 $ echo more sensible change >> file1

15 $ cat file1

16 $ git commit -am sensible

You decide that the ‘sensible’ change is the one you want to keep.

First get the reference id for that ‘sensible’ commit with a git log:

17 $ git log --all --oneline --decorate --graph

then checkout the master and run a cherry-pick command using the identifier (which you replace ‘ID’ with below):

Sometimes a cherry-pick (not the next one) might fail because the diff cannot easily be applied, as in this case (remember to replace the ID below with the commit identifier you found in the git log command above):

18 $ git checkout master

19 $ git cherry-pick ID

20 $ git log --all --oneline --decorate --graph

A case where the cherry-pick is not easily applied is outlined below. It’s exactly the same as the run-through above, but this time the ‘crazy change’ is not added at the top of the file, but in a line affected on the other branch also.

21 $ mkdir lgthw_cherry_pick_2

22 $ cd lgthw_cherry_pick_2

23 $ git init

24 $ echo change1 > file1

25 $ git add file1

26 $ git commit -am change1

27 $ echo change2 >> file1

28 $ git commit -am change2

29 $ git log --all --oneline --decorate --graph

30 $ git branch experimental

31 $ git checkout experimental

32 $ echo crazy change >> file1

That last line is the different one - you just appended it to the file rather than adding it to the top of the file like you did last time.

33 $ cat file1

34 $ git commit -am crazy

35 $ echo more sensible change >> file1

36 $ cat file1

37 $ git commit -am sensible

38 $ git log --all --oneline --decorate --graph

again, remember to replace the ID below with the commit identifier you found in the git log command above):

39 $ git checkout master

40 $ git cherry-pick ID

When cherry-picking the above you will get a message like this:

error: could not apply 743d18e... sensible

hint: after resolving the conflicts, mark the corrected paths

hint: with 'git add <paths>' or 'git rm <paths>'

hint: and commit the result with 'git commit'

in which case you need to follow the instructions above.

As ever, a git status helps you see what’s going on.

41 $ git status

On branch master

You are currently cherry-picking commit 743d18e.

(fix conflicts and run "git cherry-pick --continue")

(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:

(use "git add <file>..." to mark resolution)

both modified:   file1

no changes added to commit (use "git add" and/or "git commit -a")

At this point you should be able to figure out what to do next: resolve the conflict manually and git add the resolved file.

Cherry-picking is often a simple and easy to follow way to move changes between different branches, which can be very useful.

Cleanup

To clean up, run:

42 $ cd ..

43 $ rm -rf lgthw_cherry_pick

44 $ rm -rf lgthw_cherry_pick_2

What You Learned

- What git cherry-pick does

- How a conflict can occur when cherry-picking

Exercises

1) Clone a git repository from GitHub and create a branch in your cloned repository. Then go to that branch and randomly cherry-pick changes from the main (usually master) branch onto your branch, and explain to yourself what is going on as you do it.

2) Read the man git-cherry-pick page (or find it online) and read through it. It’s quite short. Note anything you don’t understand and research that.

Git Rebase

This section covers:

- git rebase

- What ‘fast-forward’ means in git

Rebasing is one of the most commonly-discussed advanced git topics.

In essence it’s quite simple, but it can get very confusing to the uninitiated.

Fast-forwarding is another topic that can confuse the uninitiated.

In this section you’re going to learn about rebasing and fast-forwarding with a simple example, so pay close attention to what follows!

Outline of a Simple Rebase

Let’s say you have a set of changes on a master branch:

Simple Git Repository With One Main Line

Reminder: Git Log Diagrams

git log prefers to show the history from most recent to oldest, which is the opposite of the diagrams in this section. The git man pages like to show time from left to right, like this:

A'--B'--C' topic

/

D---E---F---G master

and at this point you branch off to feature1 and make another change:

feature1 Branched Off

Now you go back to master and make a couple more changes:

Two Changes Made On Master

Now think about this from the point of view of the developer of feature1.

She has made a change from point C on the master branch, but the situation there has moved on since. Now if master wants to merge in the change on feature1, it could merge it in, and the tree would look like this:

feature1 And master Branches Merged In To G

That is OK, but not entirely desirable for two reasons:

- The history has just got more complicated (more commits)

- You have introduced an extra new change (G), which is the merge of D and F

Wouldn’t it be better if the history looked like this?

Proposed Simpler History

This is much cleaner and easier to follow. If, for example, a bug was introduced in D, it’s easier to find (eg using git bisect, which you will learn about soon). Also, the feature1 branch can be safely deleted without any significant information being lost, making the history tidier and simpler.

If you remind yourselves of the situation pre-merge (above) then you can visualise ‘picking up’ the changes on the feature1 branch and moving them to the HEAD. So from this:

Git Rebase ‘Before’

To this:

Git Rebase ‘After’

This is all a simple rebase is

What is a rebase?

A rebase is actually a more abstract concept that we cover in slightly more detail in Part IV. But for now you don’t need to worry about that. In 99% of daily discussions about rebases, this is what people mean.

Once the above is done, you’ll move master so it’s pointing at the same place as feature1:

Final Rebased State

You take a set of changes from a particular point and apply them from a different point - literally re-base your changes!

Squashing

Be aware that people also talk about rebasing to ‘squash’ commits. This is a slightly different scenario that uses the same rebase command in a slightly different way. We cover this in Part IV.

Walkthrough of a Simple Rebase

Let’s walk through the above scenario with git commands.

1 $ mkdir lgthw_rebase

2 $ cd lgthw_rebase

3 $ git init

4 $ echo A > file1

5 $ git add file1

6 $ git commit -am A

7 $ echo B >> file1

8 $ git commit -am B

9 $ echo C >> file1

10 $ git commit -am C

11 $ git checkout -b feature1

The -b flag used above is new to this book. It’s a shortcut that both creates the branch and checks it out at once. Using this means you don’t get into the situation where you create a branch but forget to explicitly check it out before committing more changes.

12 $ echo D >> file1

13 $ git commit -am D

14 $ git checkout master

15 $ echo E >> file1

16 $ git commit -am E

17 $ echo F >> file1

18 $ git commit -am F

19 $ git log --all --decorate --graph

* commit baacf6fb432967a9d404858268928278df40c7a3 (feature1)

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     D

|

| * commit cb548ab427a50028f2dbd721f4c285cbd6ad595d (HEAD, master)

| | Author: Ian Miell <ian.miell@gmail.com>

| | Date:   Wed Jun 29 19:02:09 2016 +0100

| |

| |     F

| |

| * commit 9a9a81060dd74ded8306e7c1a49400529188df70

|/  Author: Ian Miell <ian.miell@gmail.com>

|   Date:   Wed Jun 29 19:02:09 2016 +0100

|

|       E

|

* commit 44954ddfb91d96aaa3bbedab3ae7bcb47aa833be

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     C

|

* commit a63e4ff9ba95ab478a5755ed4e3c9c9bc3ddbc37

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     B

|

* commit b1fd27851324ed88caa958e2da9d7a36e24277dc

Author: Ian Miell <ian.miell@gmail.com>

Date:   Wed Jun 29 19:02:09 2016 +0100

A

You are now in this state:

Git Rebase ‘Before’

You now go to the feature1 branch and rebase:

20 $ git checkout feature1

21 $ git rebase master

First, rewinding head to replay your work on top of it...

Applying: D

Uising index info to reconstruct a base tree...

M     file1

Falling back to patching base and 3-way merge...

Auto-merging file1

CONFLICT (content): Merge conflict in file1

Failed to merge in the changes.

Patch failed at 0001 D

The copy of the patch that failed is found in:

/Users/imiell/gitcourse/tmprebase/.git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".

If you prefer to skip this patch, run "git rebase --skip" instead.

To check out the original branch and stop rebasing, run "git rebase --abort".

Read the output carefully and follow the instructions. It’s hard to follow, so don’t be concerned if you don’t get it the first time you try - keep at it.

Stuck?

If you are struggling at this point, then you need to review the section in Part I on merging and conflicts.

You don’t have to use vi to make changes, you can use any editor you like, but the next command is there to indicate that you should resolve the conflicts mentioned.

22 $ vi file1

Now that you’ve resolved the conflicts, you can continue the rebase session:

23 $ git add file1

24 $ git rebase --continue

# Applying: D

25 $ git log --all --decorate --graph

* commit eff7c3a62c8a2ce74302207db014b0db82c22d4e (HEAD, feature1)

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     D

|

* commit cb548ab427a50028f2dbd721f4c285cbd6ad595d (master)

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     F

|

* commit 9a9a81060dd74ded8306e7c1a49400529188df70

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     E

|

* commit 44954ddfb91d96aaa3bbedab3ae7bcb47aa833be

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     C

|

* commit a63e4ff9ba95ab478a5755ed4e3c9c9bc3ddbc37

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     B

|

* commit b1fd27851324ed88caa958e2da9d7a36e24277dc

Author: Ian Miell <ian.miell@gmail.com>

Date:   Wed Jun 29 19:02:09 2016 +0100

A

The state now looks like this:

Git Rebase ‘After’

Merging and ‘Fast-Forwarding’

Now the changes are in one line you can merge the feature1 branch into the master branch to move the feature1 branch pointer along. Here we return to the git merge command you looked at earlier:

26 $ git checkout master

27 $ git merge feature1

Updating cb548ab..eff7c3a

Fast-forward

file1 | 1 +

1 file changed, 1 insertion(+)

28 $ git log --all --decorate --graph

* commit eff7c3a62c8a2ce74302207db014b0db82c22d4e (HEAD, master, feature1)

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     D

|

* commit cb548ab427a50028f2dbd721f4c285cbd6ad595d

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     F

|

* commit 9a9a81060dd74ded8306e7c1a49400529188df70

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     E

|

* commit 44954ddfb91d96aaa3bbedab3ae7bcb47aa833be

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     C

|

* commit a63e4ff9ba95ab478a5755ed4e3c9c9bc3ddbc37

| Author: Ian Miell <ian.miell@gmail.com>

| Date:   Wed Jun 29 19:02:09 2016 +0100

|

|     B

|

* commit b1fd27851324ed88caa958e2da9d7a36e24277dc

Author: Ian Miell <ian.miell@gmail.com>

Date:   Wed Jun 29 19:02:09 2016 +0100

A

And you end up in your desired state:

Final Rebased State

Note that merging and rebasing are separate topics, but we return to it here because it’s easiest to understand fast-forwarding in this context

Fast-Forwarding

What’s interesting about the above is this part:

$ git merge feature1

Updating cb548ab..eff7c3a

Fast-forward

file1 | 1 +

1 file changed, 1 insertion(+)

Because the changes are in a line, no new changes need to be made - the master branch pointer merely needs to be ‘fast-forwarded’ to the same point as feature1!

The HEAD pointer, naturally, moves with the branch you’re on (master).

Cleanup

To clean up, run:

29 $ cd ..

30 $ rm -rf lgthw_rebase

What You Learned

- What a rebase is

- What ‘fast forward’ means in a git context

Exercises

1) Clone a git repository from GitHub. Create a branch and move to it, and do a few commits.

Now check out the main line (usually master) and make some more commits.

Switch back to your branch, and rebase your changes against master.

Explain to someone (or something) what you just did, and why the GitHub repository is unaffected.

2) Some people prefer merging over rebasing. Go and look for blogs and forums debating the subject. Don’t worry if a lot of the debate goes over your head. What you should take from it is that not everyone agrees on the best way to use git, and the nature and style of the debate. One day you may get involved.

If you end up thinking merging is better than rebasing then get in touch with the author!

Git Bisect

This section covers:

- git bisect

- The ^ operator

- The ~ operator

- The git show command

Bisecting is a very powerful tool for finding bugs. You won’t necessarily need it that often, but when you do it will come in very handy, and possibly make you a hero.

However, it’s not magical, and understanding what it is will help you understand where it will be useful.

When it is you create a git bisect ‘session’ and interact with the repository until you get the answer to your problem.

Bisecting - The Session At A High Level

First I’ll explain what a git bisect session might look like in the abstract, before running through it on the command line in detail.

Let’s say you have a set of 100 commits on a master branch:

Git Repository With 100 Commits

Not All Commits Shown

To make the graphs legible, I have not shown all 100 commits in a line. Where items have been skipped I’ve put a pointer indicating they are being skipped, or were skipped. eg in the above diagram, 48 commits were skipped between A2 and A50, and 50 commits were skipped between A50 and A100.

You discover a previously-unseen bug at point A100 and want to debug it. One way to do this is to read over the code, add logging etc.. This can be time-consuming, and there is another simpler way to gather information about what change caused the bug.

The git bisect command is a very useful tool for finding out where a bug was introduced. If you know where a bug was introduced, you can look at the diff of the commit that ‘caused’ it and work out what the source of the problem is.

It works by picking a start point where the bug definitely did not exist (the ‘good’ point). In this case you’ll choose point A1. Then you pick a point where the bug definitely did exist (the ‘bad’ point). In this case, that’s A100.

Rebase - A1 ‘Good’, A100 ‘Bad’, A50…?

Once the git bisect session has that information, it can hand you a version at the halfway point between the ‘good’ and ‘bad’ points and asks you to run whatever you need to run to determine whether it’s good or bad. At this point, the halfway point is A50.

If you tell it it’s ‘good’ it will mark all version at that point and before as ‘good’.

Rebase A1-A50 Marked As ‘Good’

It then repeats the process, giving you a version at the halfway point between ‘good’ and ‘bad’, asking you for its status. In this sequence, you are given A75:

Rebase - A1-A50 Marked As ‘Good’, A75…?

If you determine that this version was ‘bad’, then all the versions after it are marked as ‘bad’:

Rebase - A75-A100 Marked As ‘Bad’

This ‘binary search’ process repeats until you know which versions were good and bad. One outcome might be:

Rebase - A1-A62 ‘Good’, A63-A100 ‘Bad’

Binary Search?

Binary search is a way of searching an ordered list of items in an algorithmic way with the fewest number of ‘moves’. If are looking for a book by the author (Ian Miell) in a library that’s on a single very long shelf that’s ordered by author, you might start at the middle. If the middle book is by an author before ‘Miell’ you would move halfway towards the remainder of the shelf, then likely back half as far again, and so on, until you get to the book.

Once you know that the first ‘bad’ commit was A63, you can examine the difference between A62 and A63, and this gives you a clue as to what caused the bug.

A Real git bisect Session

Now that you understand what a bisect session is, you’re going to embed this idea by running through an actual git bisect session. If the above felt a bit abstract, then this might help make the lesson more vivid.

What you’re going to do is create a git repository with one file (projectfile). In this file you are going to add a line for each commit. The first line will be 1, the second 2, and so on until the hundredth commit which adds the line 100.

In this scenario the ‘bug’ is the line 63, but you don’t know that yet. All you know is that you can tell if the bug is in the code with this shell command:

$ if grep -w 63 projectfile

> then

>   echo BAD

> else

>   echo GOOD

which outputs BAD if the ‘bug’ is found, and GOOD if it is absent. Obviously, this is an arbitrary example - your reproduction code for a bug in a ‘real’ situation might be far more complicated and/or time-consuming.

Type this in:

1 $ mkdir -p lgthw_bisect

2 $ cd lgthw_bisect

3 $ git init

4 $ touch projectfile

5 $ git add projectfile

6 $ cat > test.sh << END

7 > if grep 63 projectfile

8 > then

9 >   echo BAD

10 > else

11 >   echo GOOD

12 > fi

13 > END

14 $ chmod +x test.sh

15 $ git add test.sh

So far so normal. You’ve created and added a file to a new git repository, and added a test script to determine if the bug is present.

The next bit of code creates 100 lines with an incrementing number in each line, each line committed in turn:

16 $ for ((i=1;i<=100;i++))

17 > do

18 > echo $i >> projectfile

19 > git commit -am "A$i"

20 > done

Now check the history of the repository is as you expect:

21 $ git log

Now you’re going to start your bisect session to find the ‘bug’. Follow the output carefully:

22 $ git bisect start

23 $ ./test.sh

24 $ git bisect bad

25 $ git status

By marking the bisect session as ‘bad’ you have told it where to end its search for the bug. Can you see where it tells you that in the git status output?

Now type this in and try and figure out what’s going on while you do so. You will want to throw in some git status and git log commands as you go through to orient yourself:

26 $ git checkout HEAD~99

27 $ git status

28 $ ./test.sh

29 $ git bisect good

30 $ git log

The first thing to point out is the checkout command. You should know what HEAD is by now. Can you figure what the ~99 does?

That’s right, it refers to the 99 commits previous to this one. We will return to this later.

Where are you now in the history?

31 $ git status

32 $ ./test.sh

33 $ git bisect good

34 $ git log

35 $ ./test.sh

36 $ git bisect bad

At this point you should be getting the idea.

So I’m going to tell you now to keep going marking commits as good or bad until you get to something that looks like this (can you spot what’s different about it before you carry on?):

37 $ git bisect bad

79583459dc6061bd91d55cfcf8c34fae845f836b is the first bad commit

commit 79583459dc6061bd91d55cfcf8c34fae845f836b

Author: Ian Miell <ian.miell@gmail.com>

Date:   Sun Jul 10 11:53:47 2016 +0100

A63

:100644 100644 aea6bd8ad6845cca3804a87230fee1b69651643d 55200b3d5d7c0e515eaccaf8465a\

295017e88249 M    projectfile

If you got lost, go to ‘Cleanup’ below and start the section again. No cheating, now.

If all was as expected, then the bisect is complete, and has reported that 79583459dc6061bd91d55cfcf8c34fae845f836b was the first bad commit (the commit identifier may differ for you).

You can get the diff between this reported commit id and its parent by using the ^ operator with diff:

38 $ git diff 79583459dc6061bd91d55cfcf8c34fae845f836b^ 79583459dc6061bd91d55cfcf8c34fa\

39 e845f836b

diff --git a/projectfile b/projectfile

index aea6bd8..55200b3 100644

--- a/projectfile

+++ b/projectfile

@@ -60,3 +60,4 @@

60

61

62

+63

The ^ operator is a handy shortcut that allows you to quickly refer to a commit’s parent.

What’s the Difference Between ~ and ^?

These two operators are used in a very similar way, so the difference between the two can be obscure.

Let’s demonstrate it in its simplest form with another worked example:

39 $ cd ..

40 $ mkdir -p lgthw_bisect_2

41 $ cd lgthw_bisect_2

42 $ git init

43 $ touch afile

44 $ git add afile

45 $ git commit -am 'Initial commit'

46 $ git branch abranch

47 $ git checkout abranch

48 $ echo 'abranch addition' >> afile

49 $ git commit -am 'abranch addition'

50 $ git checkout master

51 $ echo 'master addition 1' >> afile

52 $ git commit -am 'master addition 1'

53 $ echo 'master addition 2' >> afile

54 $ git commit -am 'master addition 2'

55 $ git merge -X ours -m merged abranch

What happened there? You created an initial commit of a single empty file, then branched to a branch called abranch. You made a change on that branch, then checked out master and made two changes there.

One thing worth explicitly calling out here is the -X ours flag in the git merge command. When merging, you can tell git to use specific strategies to help it make decisions about how to merge, and reduce the chances of having to resolve a conflict. In this case, when faced with a conflict git will favour changes made within the current branch over the one being merged in.

So you end up with a tree that looks like this:

A Simple Branched And Merged Repository

Check this yourself and compare to the output of the git log command here:

56 $ git log --oneline --graph --all

Now you will try to show all the items using ^ and ~. You’re going to have to think through the difference between ^ and ~ below by yourself. And try some commands of your own.

It’s the only way to truly learn…

57 $ git show HEAD^

58 $ git show HEAD^^

59 $ git show HEAD^2

60 $ git show HEAD~1

61 $ git show HEAD~2

62 $ git show HEAD~2^1

Did you figure it out? If not, study the diagram again and look carefully at what git show reported. Also, use git log commands to work out what’s going on.

If all else fails, then start the subsection here again, then look at the man pages, then google it to see if that makes more sense.

If it still doesn’t make sense, make a note to look at it again at a later date.

This is how I learned a lot of things in git.

Cleanup

To clean up, run:

63 $ cd ..

64 $ rm -rf lgthw_bisect*

What You Learned

- How to bisect a git repository

- The ^ and ~ operators

- The difference between HEAD^2 and HEAD~2

Exercises

1) Find a real repository on GitHub that you’re interested in, and look for an old bug. Construct a test to find that bug and then bisect to find where it was introduced.

2) With the same repository (or another one with a lot of branches), use ^ and ~ to get to specific commits you’ve found by looking for them with git log --all.

Summary

In this section you have looked at more advanced local repository management and techniques. You covered:

- The git stash command, and how it is a special kind of branch

- The git cherry-pick command, and how you can use it to copy specific commits around

- The git rebase command

- What fast-forwarding means

- How to use the git bisect command

- Using the interactive git add option to be more selective about what you are adding to the index

- What the reflog is

These more advanced techniques are what typically separates the casual user from the git master. Don’t worry if you can’t perfectly recall it all now, it was a lot to take in. But if you work with code a lot you will likely come across many situtations where these techniques are useful.

The ones I use most frequently are:

- Stashing

- Rebasing

- Cherry-picking

in that order. The others I use more rarely, and often have to look up how to do them each time. But your use cases might differ: if you are running the tests for a complex project, then you might use bisect very regularly to identify who broke the build!

Now you’ve covered the key areas for managing your local repository, you’re going to get to the distributed part of git. Working with other repositories is the most confusing part of using git, and you will gain an understanding of what’s going on that will lay the foundations of a more throrough git competency.

Part III - Remote Repository Management

Part II dealt with advanced local repository management and techniques in preparation for moving onto Part III, which looks at working with other git repositories.

Part III is where it gets really interesting. You will start to interact with other repositories, and see how changes get fetched, pulled and pushed. Specifically, you will cover:

- git push

- git fetch

- git pull

- git submodule

- Pull requests

Fetching and Pulling Content

In Part I, I emphasized the point that all git repositories are equal in status.

In this section you will begin to grasp how these ‘equal’ git repositories communicate with each other and manage their differences.

You will cover:

- git fetch

- What git pull does

By the end of it you may want to stop using git pull…

Two Git Repos - git_origin and git_cloned

You’re going to start by creating a simple git repository and then cloning it:

1 $ mkdir -p lgthw_pull

2 $ cd lgthw_pull

3 $ mkdir git_origin

4 $ cd git_origin

5 $ git init

6 $ echo 'first commit' > file1

7 $ git add file1

8 $ git commit -am file1

9 $ cd ..

10 $ git clone git_origin git_cloned

The two repositories (the folders git_origin and git_cloned) now contain identical content:

19 $ diff git_origin/file1 git_cloned/file1

However, their .git/config files differ in instructive ways.

The git_origin folder has this in its .git/config file:

11 $ cat git_origin/.git/config

[core]

repositoryformatversion = 0

filemode = true

bare = false

logallrefupdates = true

ignorecase = true

precomposeunicode = true

while the git_cloned folder has this in its .git/config file:

12 $ cat git_cloned/.git/config

[core]

repositoryformatversion = 0

filemode = true

bare = false

logallrefupdates = true

ignorecase = true

precomposeunicode = true

[remote "origin"]

url = /Users/imiell/gitcourse/git_origin

fetch = +refs/heads/*:refs/remotes/origin/*

[branch "master"]

remote = origin

merge = refs/heads/master

While the git_origin has no visibility of any remotes, the cloned one does. The git_cloned repository has a remote called ‘origin’.

Its URL is (in this case) pointed at the directory git_origin that is local on the host. URLs can also be an http:// or https:// one, or even ssh://, or git://. The last is a git-specific protocol that is rarely seen these days.

If you go to the git_cloned repository and ask it for information about its remotes:

13 $ cd git_cloned

14 $ git remote

origin

You get the name origin back.

The name origin is the default name for a remote, but it has no special meaning. It could be renamed to bitbucket, or gitlab for example.

Fetch and Push

The above remotes are divided into “(fetch)” and “(push)” actions. These relate to two different actions on remotes, ie getting changes from a remote, or pushing changes to a remote.

These actions can work against different remotes. For example, the output of git remote -v might be:

15 $ git remote -v

origin  /Users/imiell/gitcourse/one_origin (fetch)

origin  /Users/imiell/gitcourse/another_origin (push)

but I’ve never seen an example of these entries being different from one another in the wild.

The git fetch Command

First you will look at ‘fetch’ing from the remote.

The command git fetch gets the latest changes from the remote repository and copies them into the local repository, updating the local copies of any branches that have been changed.

Crucially, these changes are not mixed with your local branches, but are kept in a separate place. So if you have a master branch on the remote git repository, and a master branch on your local git repository, your local one will not be affected.

First make a change to the origin’s repository:

16 $ cd ..

17 $ cd git_origin

18 $ echo 'fetchable change' >> file1

19 $ git commit -am fetchable

Then go to the cloned repository and fetch the changes on the master branch on the origin remote (git_origin):

20 $ cd ../git_cloned

21 $ git fetch origin master

remote: Counting objects: 3, done.

remote: Total 3 (delta 0), reused 0 (delta 0)

Unpacking objects: 100% (3/3), done.

From /Users/imiell/gitcourse/git_origin

* branch            master     -> FETCH_HEAD

ceed883..056dd2c  master     -> origin/master

We did not ‘fetch’ all the changes from the remote, but specified that we wanted the changes on the remote master branch brought down by adding the branch to the commmand.

What the above output means is that the origin’s master has been brought into this repoistory’s references. This branch is now referred to locally as: origin/master.

This has not affected the local master branch at all. Check this:

22 $ git log

You can see this repository’s view of all the branches it is aware of by running this:

23 $ git branch --all

* master

remotes/origin/HEAD -> origin/master

remotes/origin/master

Here you see the local master branch, followed by the remotes/origin/HEAD pointer (remember: HEAD is a pointer to a location in the repository), which is linked to remotes/origin/master.

If you want to dig into the internals at this point, you can peek at the .git folder again:

24 $ ls .git/refs/

heads   remotes tags

which has heads, which contains references to the local branch:

25 $ cat .git/refs/heads/master

ceed883eec5a797471cd1c62365d9f2899b857c7

and similarly for remote branches:

26 $ cat .git/refs/remotes/origin/master

056dd2ce64da1e746214107b74866c375a85ffc2

Your IDs Will Differ

Remember that the IDs you see above will differ for you, as they are effectively unique to each repository.

So you’ve git fetched the remote branch, and now have a copy of it locally.

To apply the remote master’s changes to the local one you merge it just as you would for any other reference:

27 $ git merge origin/master

Updating ceed883..056dd2c

Fast-forward

file1 | 1 +

1 file changed, 1 insertion(+)

28 $ git log

commit 056dd2ce64da1e746214107b74866c375a85ffc2

Author: Ian Miell <ian.miell@gmail.com>

Date:   Tue Jun 28 18:41:41 2016 +0100

fetchable

commit ceed883eec5a797471cd1c62365d9f2899b857c7

Author: Ian Miell <ian.miell@gmail.com>

Date:   Tue Jun 28 17:30:44 2016 +0100

file1

The key point to take from all this is that pulling involved two actions: fetching, which just brings over the changes from a remote repository and caches it locally for reference, and merging, which actually messes with your local repository.

In general I prefer that rather than using git pull you do git fetch and git merge separately and keep reminding yourself of what’s going on with respect to remotes in your git repository. Once you’ve internalised that workflow, start using git pull as a convenience. If you use git pull too early there is a danger of seeing it as magical, or at least not feeling entirely sure about what’s going on.

Cleanup

To clean up, run:

29 $ cd ../..

30 $ rm -rf lgthw_pull

What You Learned

You have learned what a git pull actually does.

A git pull does a

- git fetch

followed by a

- git merge

By default, a pull fetches the mapped remote branch, and then merges it into the local branch.

We will cover what your branch locally is mapped to remotely in the next section, where you will cover remote repository management in more depth.

Exercises

1) Look up the man page for git pull and try and follow the description at the top. Make sure you try to understand every part.

2) Clone a git repository from GitHub, and fetch and merge a particular branch from it into a new branch you have created locally. Draw a diagram of what you have done. If the repository doesn’t have a branch, find one that does.

3) Draw out how two git repos model one another’s branches, and explain it to someone else.

Working With Multiple Repositories

In this section you will begin to grasp how these ‘equal’ git repositories communicate with each other and manage their differences.

You will cover:

- Creating multiple remotes

- Moving changes between multiple remotes

By the end you will have much more control over what changes can be applied where, and access to more powerful workflows when you need them.

Three Linked Repositories…

Now you are going to work with multiple repos.

You’re going to do the same as you did in the last section, but this time you will create two clones of the origin: alice_cloned and bob_cloned.

1 $ mkdir -p lgthw_remotes

2 $ cd lgthw_remotes

3 $ mkdir git_origin

4 $ cd git_origin

5 $ git init

6 $ echo 'first commit' > file1

7 $ git add file1

8 $ git commit -am file1

9 $ cd ..

10 $ git clone git_origin alice_cloned

11 $ git clone git_origin bob_cloned

Now alice_cloned and bob_cloned have git_origin as the origin remote:

12 $ cd alice_cloned

13 $ git remote -v

origin  /Users/imiell/gitcourse/git_origin (fetch)

origin  /Users/imiell/gitcourse/git_origin (push)

14 $ cd ../bob_cloned

15 $ git remote -v

origin  /Users/imiell/gitcourse/git_origin (fetch)

origin  /Users/imiell/gitcourse/git_origin (push)

Now Alice makes a change in her master branch:

16 $ cd ../alice_cloned

17 $ echo alice_change >> file1

18 $ git commit -am 'alice change'

[master 9077a48] alice change

1 file changed, 1 insertion(+)

The workflow currently looks like this:

alice_cloned <=> git_origin <=> bob_cloned

The question is: how does Bob get Alice’s change into his master branch without going to the origin?

This is a common scenario in distributed teams. If you consider that git was created for managing the codebase of the Linux operating system, it’s easy to imagine the git_origin as Linus Torvalds’ repository, Alice as a contributor and Bob as a so-called lieutenant.

So here is how:

1) ADD Alice’s repository as a remote to Bob’s

2) FETCH Alice’s updated master branch

3) MERGE Alice’s master branch into Bob’s local one

As you have already seen, steps 2) and 3) can be collapsed into a git pull, but it is more instructive to keep these separate.

1) ADD Alice’s repository as a remote to Bob’s

First, Bob needs to add Alice’s repository as a remote.

19 $ cd ../bob_cloned

20 $ git remote add alice ../alice_cloned

21 $ git remote -v

alice   ../alice_cloned/ (fetch)

alice   ../alice_cloned/ (push)

origin  /Users/imiell/gitcourse/git_origin (fetch)

origin  /Users/imiell/gitcourse/git_origin (push)

You have now linked up your repository to Alice’s, and given it the name alice.

2) FETCH Alice’s updated master branch

22 $ git fetch alice master

remote: Counting objects: 3, done.

remote: Total 3 (delta 0), reused 0 (delta 0)

Unpacking objects: 100% (3/3), done.

From ../alice_cloned

* branch            master     -> FETCH_HEAD

* [new branch]      master     -> alice/master

Alice’s master branch is now fetched to your local repository.

23 $ git branch -vv -a

* master                fdc7132 [origin/master] file1

remotes/alice/master  9077a48 alice change

remotes/origin/HEAD   -> origin/master

remotes/origin/master fdc7132 file1

3) MERGE Alice’s master branch into Bob’s local one

24 $ git merge alice/master

Updating fdc7132..9077a48

Fast-forward

file1 | 1 +

1 file changed, 1 insertion(+)

25 $ cat file1

first commit

alice_change

You may be wondering why you use alice/master and not remotes/alice/master, as the output of git branch -vv -a tells you. You can run:

26 $ git merge remotes/alice/master

which will do the same. Git assumes that the branch is a remote (presumably from seeing the */* in the branch) and adds the remotes for you.

This ‘Lieutenants’ model is one example of a git workflow. Although it was the one git was originally created for, it is still common for developers to use a traditional centralised model around a repository such as GitLab or BitBucket.

This is why people make jokes when GitHub is down. Git is designed to be a distributed source control tool, but the simplicity of depending on a central server is also a powerful and tempting model. In any case, git can support both models.

Cleanup

To clean up, run:

27 $ cd ../..

28 $ rm -rf lgthw_remotes

What You Learned

- How to add a remote repository

- How to fetch changes from the remote repository

- How to merge changes from the remote repository

- There are different git workflows for different organisation structures

You now understand what a remote repository is, how it fits in with the all git repositories are equal mantra, and are ready to look at working with remotes and branches.

This is where git repositories’ relationships become very intertwined, and your hard work thus far will pay off as you get to doing a pull request and fully grasp what’s happening while doing so!

Exercises

1) Create a git repository and clone it to a second one. Make a change on the first repository’s master branch, and fetch and merge it into the second one.

2) Create a change on both master branches from 1) that will conflict and commit them. Resolve the conflicts and make both repositories consistent.

3) Do the same as in 2) but on three copies of the same repository (ie two clones).

4) Try setting up two independent copies of the same repository (ie do not clone) and bring changes in and out on multiple branches.

Pushing Code

In this section you will look at pushing code to a remote repository. This is one of the trickier areas of git, and one that can get very confusing to people.

You will cover:

- git push

- git push failures

By the end you will have a much greater understanding of why pushes fail, and what you can do about it when they do. This is a cause of much confusion for git newbies.

Setup

You’re familiar now with git branches and remote git repositories.

In this section you’re going to familiarise yourself with how branches are managed between the two, and what exactly goes on in a push.

First, set up a simple origin git repository and clone it, just as you did before.

1 $ mkdir -p lgthw_pushing

2 $ cd lgthw_pushing

3 $ mkdir git_origin

4 $ cd git_origin

5 $ git init

6 $ echo 'first commit' > file1

7 $ git add file1

8 $ git commit -am file1

9 $ cd ..

10 $ git clone git_origin git_clone

As it stands you have no branches on either the origin or the clone other than the default (master):

11 $ cd git_origin

12 $ git branch -a -v

* master bedca8c file1

13 $ cd ../git_clone

14 $ git branch -a -v

* master                bedca8c file1

remotes/origin/HEAD   -> origin/master

remotes/origin/master bedca8c file1

15 $ cd ..

Make sure you understand why there are three lines in the second git branch output! If you don’t, start the chapter again!

Creating and Pushing Branches

Now you’re going to create a branch on the clone, do some work on it, and then push it to the remote repository.

This is a common use case, as users may experiment with different branches locally, then decide they want to share their work with others by pushing it to a commonly-accessible remote repository, eg on GitHub.

16 $ cd git_clone

17 $ git checkout -b abranch

18 $ echo 'cloned abranch commit' >> file1

19 $ git commit -am 'cloned abranch commit'

20 $ git push origin abranch

The key bit there was at the end, with the git push command. The first item after the push specifies the remote (which is origin by default) and the branch is the next item (‘abranch’ here).

Git will create a branch on the remote repository for you if one does not already exist.

Pushing to Repositories with Different Content?

You might be asking yourself at this point: what happens if both repositories have a branch with different content?

Let’s see! Type this out.

21 $ cd ..

22 $ rm -rf git_origin git_clone

23 $ mkdir git_origin

24 $ cd git_origin

25 $ git init

26 $ echo 'first commit' > file1

27 $ git add file1

28 $ git commit -am file1

29 $ cd ..

30 $ git clone git_origin git_clone

31 $ cd git_clone

32 $ git checkout -b abranch

33 $ echo 'cloned abranch commit' >> file1

34 $ git commit -am 'cloned abranch commit'

35 $ cd ../git_origin

36 $ git checkout -b abranch

37 $ echo 'origin abranch commit' >> file1

38 $ git commit -am 'origin abranch commit'

39 $ cd ../git_clone

40 $ git push origin abranch:abranch

The output of the last command will look something like this:

To /Users/imiell/tmp/git_origin

! [rejected]        abranch -> abranch (fetch first)

error: failed to push some refs to '/Users/imiell/tmp/git_origin'

hint: Updates were rejected because the remote contains work that you do

hint: not have locally. This is usually caused by another repository pushing

hint: to the same ref. You may want to first integrate the remote changes

hint: (e.g., 'git pull ...') before pushing again.

hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Read the output carefully. It tells you exactly what’s going on. Breaking it down:

Updates were rejected because the remote contains work that you do not have

locally.

The remote (the origin) has a commit (with the content: origin abranch commit) that you have no record of locally in your branch with the same name.

It goes on:

This is usually caused by another repository pushing to the same ref.

It’s correctly diagnosed the problem as another repository (git_remote) pushing to the same branch name (‘ref’) on the receiving remote. Finally, it offers some advice.

You may want to first integrate the remote changes (e.g., 'git pull ...')

before pushing again.

But by now you should know better than to git pull without thinking! Do a fetch and merge to really understand what is going on:

41 $ git fetch origin

remote: Counting objects: 3, done.

remote: Total 3 (delta 0), reused 0 (delta 0)

Unpacking objects: 100% (3/3), done.

From /Users/imiell/tmp/git_origin

* [new branch]      abranch    -> origin/abranch

Now check the branches you have locally:

42 $ git branch -v -a

* abranch                d99581a cloned abranch commit

master                 9917bcd file1

remotes/origin/HEAD    -> origin/master

remotes/origin/abranch f2be4e0 origin abranch commit

remotes/origin/master  9917bcd file1

Observe that the remotes/origin/abranch branch you now have locally (f2be4e0 origin branch commit) is different from the local abranch branch (d99581a cloned abranch commit).

To complete your manual pull, merge the remote branch into the local:

43 $ git merge remotes/origin/abranch

Auto-merging file1

CONFLICT (content): Merge conflict in file1

Automatic merge failed; fix conflicts and then commit the result.

Follow the instructions to resolve the conflict and commit the result.

Later we will cover doing a rebase here rather than a merge for a cleaner history!

The Branch Exists Only on the Remote

It is common to have a branch that exists on a remote repository, but not in your local repository. Maybe someone else pushed a branch up, or has made a pull request from a branch in that remote repository.

Type the following out to simulate that state of affairs:

44 $ cd ..

45 $ rm -rf git_origin git_clone

46 $ mkdir git_origin

47 $ cd git_origin

48 $ git init

49 $ echo 'first commit' > file1

50 $ git add file1

51 $ git commit -am file1

52 $ cd ..

53 $ git clone git_origin git_clone

54 $ cd git_origin

55 $ git checkout -b abranch

56 $ echo 'origin abranch commit' >> file1

57 $ git commit -am 'cloned abranch commit'

58 $ git branch -a

* abranch

master

59 $ cd ../git_clone

60 $ git branch -a

* master

remotes/origin/HEAD -> origin/master

remotes/origin/master

61 $ git remote -v

origin  /tmp/git_origin (fetch)

origin  /tmp/git_origin (push)

You will observe that the cloned repository has no knowledge of the abranch branch on the origin repository, even though the origin is known to the cloned repository. There’s no magic about the tracking of a remote repository, you have to trigger your repository to read the remote’s state.

To get the branch into your repository you will need to fetch it.

62 $ git fetch origin

remote: Counting objects: 3, done.

remote: Total 3 (delta 0), reused 0 (delta 0)

Unpacking objects: 100% (3/3), done.

From /Users/imiell/tmp/git_origin

* [new branch]      abranch    -> origin/abranch

Note that you didn’t need to specify a branch to get from the origin. By default git will get all branches that may be of interest.

63 $ git branch -a

* master

remotes/origin/HEAD -> origin/master

remotes/origin/abranch

remotes/origin/master

Now your cloned repository has knowledge that a branch called abranch exists on the origin remote. But there is no branch in your local repository:

64 $ git branch

* master

Now if you check out an abranch branch in your local repository, git is smart enough to match the name and uses this branch to ‘track’ the remote branch from the origin:

65 $ git checkout abranch

Branch abranch set up to track remote branch abranch from origin.

Switched to a new branch 'abranch'

66 $ git branch -a -vv

* abranch                19a1fe0 [origin/abranch] cloned abranch commit

master                 05d6bd2 [origin/master] file1

remotes/origin/HEAD    -> origin/master

remotes/origin/abranch 19a1fe0 cloned abranch commit

remotes/origin/master  05d6bd2 file1

Pay close attention to branch tracking, as it can be very confusing to git newcomers!

Now if you git push any changes on this branch, git will attempt to push those changes to the tracked branch, ie the abranch branch on the remote repository.

Tracking Remote Branches with Different Names

More rarely, you may want to track a branch on the remote repository that has a different name. Or, you may want to manually mark the local branch as tracking a specific remote one.

In these situations, you might see this kind of error when you push:

67 $ git push

fatal: The current branch abranch has no upstream branch.

To push the current branch and set the remote as upstream, use

git push --set-upstream origin abranch

As is often the case, careful reading of the error will tell you what you need to know. It’s just the jargon that can be difficult to follow!

In this case, the error is telling you that your branch is not tracking any remote branch, so it doesn’t know what to push to.

Type in these commands to reproduce this situation:

68 $ rm -rf git_origin git_clone

69 $ mkdir git_origin

70 $ cd git_origin

71 $ git init

72 $ echo 'first commit' > file1

73 $ git add file1

74 $ git commit -am file1

75 $ cd ..

76 $ git clone git_origin git_clone

77 $ cd git_clone

78 $ git checkout -b abranch

79 $ echo 'origin abranch commit' >> file1

80 $ git commit -am 'cloned abranch commit'

81 $ git push

fatal: The current branch abranch has no upstream branch.

To push the current branch and set the remote as upstream, use

git push --set-upstream origin abranch

Now, let’s have a look at the branches you have locally when you try to push:

82 $ git branch -vv

* abranch 179b22a cloned abranch commit

master  41ffa8a [origin/master] file1

While the master branch is tracking the origin/master branch (ie the master branch on the origin remote), the abranch branch is not tracking any remote branch at all.

At this point you could run either:

git push --set-upstream origin abranch

or

git push -u origin abranch

and that would set up the tracking for you while pushing.

Before that though, you’re going to type:

83 $ git push origin abranch

Counting objects: 3, done.

Writing objects: 100% (3/3), 273 bytes | 0 bytes/s, done.

Total 3 (delta 0), reused 0 (delta 0)

To /Users/imiell/tmp/git_origin

* [new branch]      abranch -> abranch

Can you see the difference?

That successfully pushed the change to the remote branch, which was created as it did not already exist. However, if you re-run the branch command again:

84 $ git branch -vv

* abranch 179b22a cloned abranch commit

master  41ffa8a [origin/master] file1

it is still not tracking the origin’s master branch. If you add the --set-upstream / -u flag on a push, the branch will track the remote’s branch:

85 $ git push -u origin abranch

Branch abranch set up to track remote branch abranch from origin.

Everything up-to-date

86 $ git branch -vv

* abranch 179b22a [origin/abranch] cloned abranch commit

master  41ffa8a [origin/master] file1

Cleanup

To clean up, run:

87 $ cd ..

88 $ rm -rf lgthw_pushing

What You Learned

In this section you added to your knowledge about git commit. You:

- Created a branch and pushed it to a remote branch

- Tried to push to a remote with different content

- How to manage branches from a remote repository locally, and vice versa

- What branch tracking is

- What an upstream repository is

Exercises

1) Create a repository on Github.

2) Add content to it.

3) Clone the repository, create a branch, and push it remotely.

4) View the branch on GitHub.

Git Submodules

This section covers:

- git submodule

- The ^ operator

- The ~ operator

Submodules are a useful concept, and often seen in real projects.

Git submodules can be very confusing if you stumble into them without much preparation or experience. Following this tutorial, you should have a good understanding for a simple submodule workflow and what is going on when you run the core submodule commands.

Submodules allow you to loosely link different git repositories together so that they are bundled together. At the same time, it ensures that changing one will not break the other.

Sometimes you want to ‘include’ one repository in another, but not simply copy it over and have to maintain its changes separately. Submodules allow you to manage the separate codebase within your repository without affecting the other repository.

If this wasn’t clear re-read the above slowly. If it’s still not clear, follow the example below and see if it makes more sense then.

A Worked Example

Let’s say Alice maintains a library:

1 $ mkdir -p lgthw_submodules

2 $ cd lgthw_submodules

3 $ mkdir alicelib

4 $ cd alicelib

5 $ git init

6 $ echo 'A' > file1

7 $ git add file1

8 $ git commit -am 'A'

9 $ git checkout -b experimental      # Branch to experimental

10 $ echo 'C - EXPERIMENTAL' >> file1

11 $ git commit -am EXPERIMENTAL

12 $ git checkout master

13 $ echo 'B' >> file1

14 $ git commit -am 'B'

Alice’s library’s history looks like this:

Alice’s Library’s History

Now Bob wants to use Alice’s library for his own code, but specifically wants to use what’s on the experimental branch.

One option is to copy the code over directly, but that seems to be against the spirit of git.

If an improvement is made on the experimental branch, or Bob wants to move later to follow what’s on the master branch, then he must copy over the code he wants. For one file it might be manageable, but for a more realistic and large project, managing this will be completely impractical.

Another option is to check out the code in another folder and link to it in some predictable way in the code (eg your code might run source ../alice_experimental). Again, this causes management problems, as the user checking out the source must remember to keep code outside this git repository in a certain place for it all to work.

The git submodule Command

Git submodules solve these ‘external repository’ dependency issues, with a little overhead. Now that you understand local and remote repositories, it will be much easier to grasp how submodules work.

Git submodule commands are git commands used to track copies of other repositories within your repository. The tracking is under your control (so you decide when it gets ‘updated’, regardless of how the other repository moves on), and the tracking is done within a file that is stored with your git repository.

Pay Attention Here!

Git submodules can be very confusing if you don’t follow a few basic patterns or understand how they work, so it’s worth paying attention to this.

Let’s make this clearer with a walkthrough.

You are going to assume you have the alicelib repository created as above.

Now create Bob’s repository (‘bob_repo):

15 $ cd ..

16 $ mkdir bob_repo

17 $ cd bob_repo

18 $ git init

19 $ echo 'source alicelib' > file1

20 $ git add file1

21 $ git commit -am 'sourcing alicelib'

22 $ echo 'do something with alicelib experimental' >> file1

23 $ git commit -am 'using alicelib experimental'

24 $ cat file1

source alicelib

do something with alicelib

Now you have Alice’s repository referenced in bob_repo’s code, but bob_repo has no link to alice_repo’s code.

The first step to including alicelib in bob_repo is to initialise submodules:

25 $ git submodule init

Once a git submodule init has been performed, you can git submodule add the submodule you want:

26 $ git submodule add ../alicelib

Cloning into 'alicelib'...

done.

A new file has been created (.gitmodules), and the folder alicelib has been created:

27 $ ls -a

.               ..              .git            .gitmodules     alicelib        file1

alicelib has been cloned just as any other git repository would be anywhere else:

28 $ ls -a alicelib/

.       ..      .git    file1

but the .gitmodules file tracks where the submodule comes from:

29 $ cat .gitmodules

[submodule "alicelib"]

path = alicelib

url = ../alicelib

If you get confused, git provides a useful git submodule status command that works in a similar way to the standard git status command:

30 $ git submodule status

ff75b7fc52c3a7d52d89a47fd27d7d22ed280b6f alicelib (heads/master)

Now, you may have some questions at this point, such as:

- How do you get to the experimental branch?

- What happens if Alice’s branch changes? Does my code automatically update?

- What if I make a change to alicelib within my repository submodule checkout? Can I push those to Alice’s? Can I keep those private to my repository?

- What if there are conflicts between these repositories?

and so on. I certainly had these questions when I came to git submodules, and with some trial and error it took me some time to understand what was going on, so I really recommend playing with these simple examples to get the relationships clear in your mind.

Get the Experimental Branch

Since your alicelib submodule is a straightforward clone of the remote alicelib origin, you have the master branch and the origin’s experimental branch:

31 $ cd alicelib

32 $ git branch -a -vv

* master                      ff75b7f [origin/master] B

remotes/origin/HEAD         -> origin/master

remotes/origin/experimental 969b840 C EXPERIMENTAL

remotes/origin/master       ff75b7f B

You are on the master branch (current HEAD location is indicated with a ‘*’), which is mapped to remotes/origin/master.

References Will Be Different

the refs (eg ff75b7f) will be different in your output

You do not have an experimental branch locally. However, if you checkout a branch that does not exist locally but does exist remotely, git will assume you want to track that remote branch.

33 $ git checkout experimental

Branch experimental set up to track remote branch experimental from origin.

Switched to a new branch 'experimental'

34 $ git branch -a -vv

* experimental                969b840 [origin/experimental] C EXPERIMENTAL

remotes/origin/HEAD         -> origin/master

remotes/origin/experimental 969b840 C EXPERIMENTAL

remotes/origin/master       ff75b7f B

More Than One Remote With The Same Branch?

If more than one remote has the same name, git will not perform this matching. In that case you would have to run the full command that follows.

Alternatively, you could track a completely different branch if you specify it:

35 $ git checkout -b alicemaster --track origin/master

assuming it’s the origin’s master branch that you want to track.

Git Tracks the Submodule’s State

Now that you’ve checked out and tracked the remote experimental branch in your submodule, a change has taken place in bob_repo. If you return to bob_repo’s root folder and run git diff you will see that the subproject commit of alicelib has changed:

35 $ cd ..

36 $ git diff

diff --git a/alicelib b/alicelib

index ff75b7f..969b840 160000

--- a/alicelib

+++ b/alicelib

@@ -1 +1 @@

-Subproject commit ff75b7fc52c3a7d52d89a47fd27d7d22ed280b6f

+Subproject commit 969b840142f389de55357350a6f26f0825e02393

The commit reference now matches the experimental reference.

Note that bob_repo tracks the specific commit and not the remote branch. This means that changes to alicelib in the origin repository are not automatically tracked within bob_repo’s submodule. This is a crucial difference with submodules: they are ‘static’ in that they are pointed at a specific point in the history until you move it on.

You want to commit this change to the submodule:

37 $ git commit -am 'alicelib moved to experimental'

[master 1f67953] alicelib moved to experimental

2 files changed, 4 insertions(+)

create mode 100644 .gitmodules

create mode 160000 alicelib

Alice Makes a Change

Alice now spots a bug in her experimental branch that she wants to fix:

38 $ cd ../alicelib

39 $ git checkout experimental

40 $ echo 'D' >> file1

41 $ git commit -am 'D - a fix added'

Now there is a mismatch between alicelib’s experimental branch and bob_repo’s experimental branch.

42 $ cd ../bob_repo/alicelib

43 $ git status

On branch experimental

Your branch is up-to-date with 'origin/experimental'.

nothing to commit, working directory clean

git status reports that bob_repo’s alicelib is up-to-date with origin/experimental. Remember that origin/experimental is the locally-stored representation of alicelib’s experimental branch. Since you have not contacted alicelib to see if there are any updates, this is still the case.

To get the latest changes you can perform a fetch and merge, or save time by running a git pull, which does both:

44 $ git pull

remote: Counting objects: 3, done.

remote: Total 3 (delta 0), reused 0 (delta 0)

Unpacking objects: 100% (3/3), done.

From /Users/imiell/gitcourse/alicelib

969b840..1a725f6  experimental -> origin/experimental

Updating 969b840..1a725f6

Fast-forward

file1 | 1 +

1 file changed, 1 insertion(+)

Warning!

Generally I would advise not editing repositories that are checked out as submodules until you are more experienced with git. You quickly may find yourself in a         detached HEAD state and confused about what you’ve done.

Cloning a Project with Submodules

Submodules have a special status within git repositories. Since they are both included within a repository and at the same time reference a remote repository, a simple clone will not check out the included submodule. Let’s show that:

45 $ cd ../..

46 $ git clone bob_repo bob_repo_cloned

47 $ cd bob_repo_cloned

48 $ ls -1

alicelib

file1

49 $ cd alicelib

50 $ ls

51 $ cd ..

alicelib’s content is not there. Confusingly, git submodule status gives you little clue what’s going on here.

52 $ git submodule status

-969b840142f389de55357350a6f26f0825e02393 alicelib

The dash (or minus sign) at the front indicates the submodule is not checked out. Only by running a git submodule init and a git submodule update can you retrieve the appropriate submodule repository:

53 $ git submodule init

Submodule 'alicelib' (/Users/imiell/gitcourse/alicelib) registered for path 'aliceli\

b'

54 $ git submodule update

Submodule path 'alicelib': checked out '969b840142f389de55357350a6f26f0825e02393'

55 $ git submodule status

969b840142f389de55357350a6f26f0825e02393 alicelib (969b840)

Now the submodule status has no dash, and a commit ID has been added to the output (969b840…).

The git clone --recursive Flag

Fortunately there is an easier way. You can clone the repository with a --recursive flag to automatically init and update any submodules (and submodules of those submodules ad infinitum) within the cloned repository:

56 $ cd ..

57 $ git clone --recursive bob_repo bob_repo_cloned_recursive

Cloning into 'bob_repo_cloned'...

done.

Submodule 'alicelib' (/Users/imiell/gitcourse/alicelib) registered for path 'aliceli\

b'

Cloning into 'alicelib'...

done.

Submodule path 'alicelib': checked out '969b840142f389de55357350a6f26f0825e02393'

In case you’re wondering, if you clone a repository with --recursive, and there are no submodules, then git does not complain.

Cleanup

To clean up, run:

58 $ rm -rf lgthw_submodules

What You Learned

- How to set up git submodules

- How to add a submodule to a repository

- How to track remote branches

- How to checkout submodules with init and update

- How to checkout submodules with recursive

Exercises

1) Create two git repositories that have each other as submodules.

Pull Requests

This section builds on the knowledge you’ve gained so far to cover:

- Pull requests

- GitHub pull requests

What Are Pull Requests?

In essence pull requests are very simple, but they can get confusing to newbies because of all the other concepts that are related and can complicate discussion about them.

Fortunately you’ve covered them, so you are ready. They are:

- Remotes

- Branches

- Repository relationships

- Reconciling remote branches

A pull request is a request from a user for another user to accept a change that has been committed elsewhere.

This request can come in any form at all that makes sense. You can send an email with the diffs to the maintainer, fork and branch, then send a reference to the branch, branch on the maintainer’s repository and mail them the branch name, put a request in plain English on a post-it - whatever works!

I’m going to focus here on the standard GitHub pull request model.

Pull Requests Are Not ‘Core’ Git

The GitHub pull request is not necessarily identical to other applications’ (or workflows’) pull request methods. Usually issues don’t come up, but remember that details of the process can differ between them.

GitHub Pull Requests

For this section we will be looking specifically at the GitHub model of pull requests. Not all pull requests and relationships between repositories are managed in exactly the same way among different git servers.

The standard GitHub model is to:

- Fork the repository

- Make a branch on the forked repository

- Make changes on this branch

- Make a pull request to the original repository to merge this branch’s changes

Your task now is to do this on the GitHub repository!

There is a file called records/trained_users.txt in this repository. You’re going to add your name to it and raise that change as a pull request.

Remember that this is just one model of pull request! We won’t cover details on the others, just be aware that they can differ so you are not surprised in future.

If you haven’t created a GitHub account, please do so now. It’s free. Go to https://github.com and follow the instructiotns to sign up.

Fork the Repository

Next you need to fork the repository. To do this, go to the learngitthehardway repository on GitHub:

https://github.com/ianmiell/learngitthehardway

and click on the Fork button near the top.

You will now have created a fork of the repository in your own account. Replace YOURUSERNAME with your username in the below URL and you should see the same repository homepage:

https://github.com/YOURUSERNAME/learngitthehardway

Branch on the Forked Repository

To make a branch on your forked repository, type in these commands:

1 $ git clone https://github.com/YOURUSERNAME/learngitthehardway

2 $ cd learngitthehardway

3 $ git checkout -b myfirstbranch

4 $ git status

You just cloned your forked version of the repository, and created a branch called myfirstbranch. As ever, running git status gives you a quick view of which branch you’re on.

Make Change on the Branch

Now type in these commands to make a change and push it to GitHub:

5 $ echo 'my change to the README' >> README.md

6 $ git commit -am 'my change to the README'

7 $ git push -u origin myfirstbranch

The first command adds a line to the README.md file. The second commits the change you made to this new branch.

Understand the Relationships

Make sure these relationships are clear in your mind!

- The ianmiell/shutit repository was forked to YOURUSERNAME/shutit

- You cloned the repository YOURUSERNAME/shutit to a local repository /path/to/shutit

- Within this local repository, you created a branch myfirstbranch

- On this branch, you commits a change with the comment my change to the README

Draw the above out in your own diagram to help you.

Specify Remote Branch

Another way to push your branch (and one that may make the relationship clearer) is the following:

8 $ echo 'another change to the README' >> README.md

9 $ git commit -am 'another change to the README'

10 $ git push -u origin myfirstgitbranch:myfirstgitbranch

What’s changed here is that we have added myfirstgitbranch:myfirstgitbranch to the branch argument of the command.

What this does is indicate that the local branch myfirstgitbranch should be pushed to the remote branch myfirstgitbranch. The colon separates the two branch names. The first is the local one, and the second the remote one.

Of course, in this case the branch names are the same (myfirstgitbranch), but this need not be the case. By default, git assumes you want to match the names on the local and the remote repository, but it’s useful to get into the habit of typing the full specification with the colon, because there are times when it’s useful to know that this mapping is possible.

To take advantage of the defaults, the last command could have been written like this:

$ git push -u origin myfirstgitbranch:

Delete A Remote Branch

You can use a similar trick to delete a remote branch.

To practice this, create a tmpbranch on your local and remote repository.

11 $ git branch tmpbranch

12 $ git checkout tmpbranch

13 $ echo 'a temp change on tmpbranch' >> README.md

14 $ git commit -am 'a temp change on tmpbranch'

15 $ git push origin tmpbranch:tmpbranch

Now that you’ve created the tmpbranch on the remote repository, you might decide you’ve been too hasty, and that tmpbranch is not needed on the remote.

To delete it on the remote, you specify nothing before the colon, like this:

16 $ git push origin :tmpbranch

This has the effect of removing the branch on the remote repository. If you look, it’s still there on your local repository, so nothing has been lost.

Quite often, projects on GitHub can accumulate a lot of branches, and this method can be a handy quick way to tidy up these branches.

Make Pull Request

Now you have a branch on the forked repository on GitHub, you want to get that branch’s changes into the maintainer’s repository. This is where you raise the pull request.

Go to GitHub in a browser and view your repository:

https://github.com/YOURUSERNAME/learngitthehardway

The instructions for creating a pull request are here:

https://help.github.com/articles/creating-a-pull-request/

I won’t repeat it here, because the workflow can change. But in essence, the general process is to:

- Go to your branch

- Generate a new pull request

- Fill out the form

- Wait

- Celebrate your PR’s acceptance into the code, or chase the maintainer (nicely!) for an update

You can create a pull request across forks (a request to the upstream (original) repository) or against another branch in a GitHub repository. Across forks is what’s most commonly meant by a public GitHub PR, a request to accept a change made to a repository under your control to a repository under someone else’s (usually more senior to the project).

Pull Requests in Practice - Rebasing

Maintainers will often ask that you rebase your branch to the principal branch (usually master) before making a pull request.

You will remember the git rebase from Part II. If you don’t remember, you might want to go back and read over it again.

Maintainers will want you to rebase, so that the work of merging any changes made since you forked from the origin is done by you, the submitter, rather than them. This also makes the history of the main line easier.

If you didn’t understand the above paragraph, then definitely work through the rebase section again!

The goal is that all the messy work is done on your secondary branch (which in git is a more disposable thing) and the good stuff makes its way into the main line. Many projects will delete branches once they have served their purpose, and git supports this.

17 $ git branch -d mybranch

It will even warn you if the branch has not been merged into the branch you are currently on!

18 $ git branch -d abranch

error: The branch 'abranch' is not fully merged.

If you are sure you want to delete it, run 'git branch -D abranch'.

Cleanup

You might not want to clean this one up, as you may be in the process of waiting for your pull request to be accepted.

If you do want to clean up, run:

19 $ cd ..

20 $ rm -rf learngitthehardway

What You Learned

In this section you’ve finally got to a key part of git culture. Pull requests are talked about everywhere, and it’s vital that you get comfortable with what they are if you are going to collaborate with others.

You’ve also snuck in a useful bit of knowledge about deleting remote branches, and mentioned the importance of rebasing again.

The best thing you can do at this point for your development is start using git in anger on a real project. If you can’t find one, feel free to interact with the author on the project that contains this book:

https://github.com/ianmiell/learngitthehardway

Exercises

1) Submit a pull request to this repository (https://github.com/ianmiell/learngitthehardway) and see what happens!

2) Create a branch on your local repository and map it to a branch on the remote repository.

3) Delete the remote repository branch that you have mapped in 2).

4) Delete the local branch that you have created in 2).

Summary

In this section you’ve taken a step outside your local repository and started interacting with other repositories. This is where git really gets interesting, as changes can be made and moved between different locations.

You’ve also learned about submodules, which allow git repositories to be nested inside one another.

Finally, you’ve got to grips with what a pull request is, a central concept within git usage.

You’re already way beyond most git users’ understanding of what’s going on, and going to build up to an even deeper understanding in the next chapter, where you grapple with even more advanced topics.

Part IV - Advanced Git

Part III took you to the point where you could understand everything needed to perform and manage pull requests.

Part IV deals with more advanced topics that are not necessary for competent everyday git usage, but are useful to know for an advanced understanding or more advanced adminstration or problem-solving.

In Part IV you will cover:

- git log (advanced usage)

- git rev-list

- git rebase -i, or squashing commits

- ‘Bare’ repositories

- Limitations of git cherry-pick and three-way merges

- ‘Patch and apply’ing commits to other branches

In Part IV, the training wheels will be taken off. You’re going to get more instructions to go and study while you are guided through the topics. You should be at the point now where you’re comfortable starting to figure out man pages yourself.

This is not to say that it will be easy for you - expect it to be hard. But now you’re getting to the point where you will need to go and face the challenges yourself without a book, and I want you to be ready.

Go to it!

Git Log

Git contains within each repository’s local database a history of the changes made.

While many git-based tools will give you a visualisation of this history to help you review the state of the system, it is less well-known that the git log command has many useful flags which give you the power to view the history in whatever way you like.

A familiarity with the git log command can not only save you a lot of time, it can speed up the git learning process, and is available wherever git is, so you’re not dependent on a particular tool to use git effectively.

A Realistic Log History

So far you’ve seen fairly contrived and straightforward git repositories.

First, you’re going to clone a codebase with a past. My rather messy shutit project is a good example to look at. Not because of any careful curation or attention to detail in its construction, but rather for the messiness of its history as I’ve become better acquainted with git.

1 $ git clone https://github.com/ianmiell/shutit.git

2 $ cd shutit

Once cloned (it can take a while) run this command to view the kind of output git log can show you. Type this out:

3 $ git log --graph --oneline --all --decorate --topo-order

* 2ccd639 (HEAD -> asciinema, origin/master, origin/HEAD, master) whoami

* bef09f4 0.9.230

* 945cfb8 check_exit

* ddba50f expect

* 97bb290 delaybeforesend

* 7281409 remove print

* dd82f13 cygwin

* 8365e72 add delaybeforesend

* 6c840dc 0.9.228

* 37defe7 base

* 4fbea61 only once, and len of b64 contents

* 9d6a6ea add maxread, but switch off from send_file; add warning about speed

* 0381298 stray comma

* c8c09ff add timeout

* 5577a87 curl -L to follow links, and do not assume bash is ok to send locally

* dcbb99f latest

* 904b83e tpyo

* 2ca2fc6 check_last_exit_values calling bug

*   235136a Merge branch 'master' of github.com:ianmiell/shutit

|\

| * f1c55f3 exit

* |   9c4e316 Merge branch 'master' of github.com:ianmiell/shutit

|\ \

| |/

| * 0907da1 handle int

| * 088dbf5 increase timeout

* |   c307653 Merge branch 'master' of github.com:ianmiell/shutit

|\ \

| |/

| * 9314fed (testing) telemetry improved

[...]

By the end of this chapter you’ll understand what’s going on here.

git log

We have already seen git log earlier in the course.

The output is most-recent commit first, down to the oldest from the current branch.

4 $ git log

commit 2ccd639a3ffc8050b03315eb58153d9cc81831d0

Author: Ian Miell <ian.miell@gmail.com>

Date:   Thu Oct 20 08:44:36 2016 +0100

whoami

commit bef09f4ef17173c4d24ade4f85ece058fc36ccf1

Author: Ian Miell <ian.miell@gmail.com>

Date:   Wed Oct 19 16:34:04 2016 +0100

0.9.230

commit 945cfb8913750aff34d9cbcd91b650a05b82dde3

Author: Ian Miell <ian.miell@gmail.com>

Date:   Wed Oct 19 15:44:52 2016 +0100

check_exit

[...]

git log --oneline

Most of the time I don’t care about the author or the date, so in order that I can see more per screen, I use –oneline to only show the commit id and comment per-commit.

5 $ git log --oneline

2ccd639 whoami

bef09f4 0.9.230

945cfb8 check_exit

ddba50f expect

97bb290 delaybeforesend

7281409 remove print

dd82f13 cygwin

8365e72 add delaybeforesend

6c840dc 0.9.228

37defe7 base

4fbea61 only once, and len of b64 contents

9d6a6ea add maxread, but switch off from send_file; add warning about speed

0381298 stray comma

c8c09ff add timeout

5577a87 curl -L to follow links, and do not assume bash is ok to send locally

dcbb99f latest

[...]

git log --graph

The problem with the above is that you only see a linear series of commits from the HEAD, and get no sense of what was merged in where.

6 $ git log --oneline --graph

* 2ccd639 whoami

[...]

* 2ca2fc6 check_last_exit_values calling bug

*   235136a Merge branch 'master' of github.com:ianmiell/shutit

|\

| * f1c55f3 exit

* |   9c4e316 Merge branch 'master' of github.com:ianmiell/shutit

|\ \

| |/

| * 0907da1 handle int

| * 088dbf5 increase timeout

* |   c307653 Merge branch 'master' of github.com:ianmiell/shutit

[...]

You can see where merges take place, and what commits were merged. If you recall the section on rebasing, this is why it is preferable to rebase your branches to the main branch and fast-forward on a merge - you get a cleaner and simpler history.

git log -–all

By default you only get the history leading up to the HEAD (ie where you are currently in the git history. Often I want to see all the branches in the history, so I add the -–all flag.

7 $ git log --graph --oneline --all

Note that for this repository the output might look very similar to the previous one. If you want to prove there’s a difference, try this:

8 $ git log --graph --oneline --all > all_output

9 $ git log --graph --oneline > noall_output

10 $ diff all_output noall_output

41,45d40

< | *   62ce794 WIP on master: 6561888 0.9.221

< | |\

< |/ /

< | * a10d7d5 index on master: 6561888 0.9.221

< |/

99,106d93

< | * 63bb0b0 keys

< | * e4287a6 bytes

< | * c050cb5 bytes

< | * bdb1104 bytes

< | * 41b3248 at end of file

< | * 408a013 bytes

< | * c764366 python3 work

< |/

258a246

> | * 133c137 test

[...]

11 $ rm all_output noall_output

git log -–decorate

That’s great, but I can’t see what branch is where! This is where you use the -–decorate flag.

12 $ git log --graph --oneline --all --decorate

* 2ccd639 (HEAD -> asciinema, origin/master, origin/HEAD, master) whoami

[...]

* 2ca2fc6 check_last_exit_values calling bug

*   235136a Merge branch 'master' of github.com:ianmiell/shutit

|\

| * f1c55f3 exit

* |   9c4e316 Merge branch 'master' of github.com:ianmiell/shutit

|\ \

| |/

| * 0907da1 handle int

| * 088dbf5 increase timeout

* |   c307653 Merge branch 'master' of github.com:ianmiell/shutit

|\ \

| |/

| * 9314fed (testing) telemetry improved

| * 0492bff unfinished testing class

* | 0ac55d7 tests updated

|/

[...]

* f9eb51a #267: eradicate delaybeforesend

| *   62ce794 (refs/stash) WIP on master: 6561888 0.9.221

| |\

|/ /

[...]

Now you’re cooking with gas! Each remote or type of branch/tag is shown in a different colour (even stashes!). On my terminal, remotes are in red, HEAD is blue, local branches are in green, stashes in pink.

If you want, you can show the ref name on each line by adding -–source, but I usually find this to be overkill:

13 $ git log --graph --oneline --all --decorate --source

* 2ccd639       refs/heads/asciinema (HEAD -> asciinema, origin/master, origin/HEAD,\

master) whoami

* bef09f4       refs/heads/asciinema 0.9.230

[...]

*   235136a     refs/heads/asciinema Merge branch 'master' of github.com:ianmiell/sh\

utit

|\

| * f1c55f3     refs/heads/asciinema exit

* |   9c4e316   refs/heads/asciinema Merge branch 'master' of github.com:ianmiell/sh\

utit

|\ \

| |/

| * 0907da1     refs/heads/asciinema handle int

| * 088dbf5     refs/heads/asciinema increase timeout

* |   c307653   refs/heads/asciinema Merge branch 'master' of github.com:ianmiell/sh\

utit

|\ \

| |/

| * 9314fed     refs/heads/testing (testing) telemetry improved

| * 0492bff     refs/heads/testing unfinished testing class

* | 0ac55d7     refs/heads/asciinema tests updated

|/

* 90c78b1       refs/heads/testing tidier

[...]

| *   62ce794   refs/stash (refs/stash) WIP on master: 6561888 0.9.221

[...]

git log -–simplify-by-decoration

If you’re looking at the whole history of your project, you may want to see only the significant points of change (ie the lines affected by -–decorate above) to eliminate all the intermediary commits. This is perfect for getting an overview of the project’s shape as a whole.

14 $ git log --graph --oneline --all --decorate --simplify-by-decoration

* 2ccd639 (HEAD -> asciinema, origin/master, origin/HEAD, master) whoami

* 9314fed (testing) telemetry improved

| * 62ce794 (refs/stash) WIP on master: 6561888 0.9.221

|/

* 2bc6afa (origin/testing) a.py

| * 63bb0b0 (origin/python3) keys

|/

* a3c1b23 (origin/development) remove refs to base image

* 9b4c569 (development) 0.9.206

| * 1918650 (origin/gh-pages, gh-pages) shutit

| | * 7d8a826 (origin/run) started

| |/

|/|

| | * ca370a4 (origin/test) hash

| |/

|/|

| | * ea5540d (origin/assert) latest tests

| |/

|/|

| | * d006066 (origin/shutitfile) merged

| |/

|/|

* | e617184 (origin/pexpect_object) appears to work

* | 94387f1 (origin/logging) logging

* | d1dbd43 (origin/templates) Merge branch 'templates' of github.com:ianmiell/shuti\

t into templates

* | 83a4530 (origin/241) only on interactive

* | deea5aa (origin/239) try and move to correct dir

* | 6564f06 (origin/depends) depends

* | dbe5aec (origin/training_after) arg added

* | 049e4e3 (origin/volumes) volumes

* | 20e2770 (origin/expect_error) look in root library folder also

| | * 6bd6eb2 (origin/haproxy) quotes

[...]

git log -–pretty

When viewing the whole history of the project in this way, you might want to re-introduce the Date info with the -–pretty=* flag:

15 $ git log --graph --oneline --all --decorate --simplify-by-decoration --pretty='%ar \

16 %s %h'

* 3 days ago whoami 2ccd639

* 2 weeks ago telemetry improved 9314fed

| * 3 weeks ago WIP on master: 6561888 0.9.221 62ce794

|/

* 3 weeks ago a.py 2bc6afa

| * 4 weeks ago keys 63bb0b0

|/

* 3 months ago remove refs to base image a3c1b23

* 3 months ago 0.9.206 9b4c569

| * 3 months ago shutit 1918650

| | * 4 months ago started 7d8a826

| |/

|/|

| | * 5 months ago hash ca370a4

| |/

|/|

| | * 5 months ago latest tests ea5540d

| |/

|/|

| | * 5 months ago merged d006066

| |/

|/|

* | 6 months ago appears to work e617184

* | 7 months ago logging 94387f1

* | 7 months ago Merge branch 'templates' of github.com:ianmiell/shutit into templat\

es d1dbd43

* | 7 months ago only on interactive 83a4530

* | 10 months ago try and move to correct dir deea5aa

* | 11 months ago depends 6564f06

* | 11 months ago arg added dbe5aec

* | 1 year, 1 month ago volumes 049e4e3

* | 1 year, 2 months ago look in root library folder also 20e2770

| | * 1 year, 2 months ago quotes 6bd6eb2

| |/

|/|

* |   1 year, 3 months ago merged 2ae6568

|\ \

| * \   2 years, 2 months ago Merge branch 'gh-pages' of https://github.com/ianmiell\

/shutit da82df7

[...]

This gives a formatted output, showing (in this case) the relative timestamp (%ar), the commit subject (%s), and the short hash (%h).

You can even see the abstract shape of the git repository by not printing any details!

17 $ git log --graph --oneline --all --decorate --simplify-by-decoration --pretty=

* * | *

|/  * | *

|/  * * | * | | *

| |/

|/|   | | *

| |/

|/|   | | *

| |/

|/|   | | *

| |/

|/|   * | * | * | * | * | * | * | * | * | | | *

| |/

|/|   * |

|\ \  | * \

| |\ \

| | |/  | | | *

| |_|/

|/| |   | | | *

| |_|/

|/| |   | | | *

| |_|/

|/| |   * | | * | |

|\ \ \

| |/ /

|/| /

| |/    | * *

The output of this seems to vary by git version, so it might look different to you.

Cleanup

To clean up, delete the repository you cloned:

18 $ cd ..

19 $ rm -rf shutit

What You’ve Learned

In this section you’ve gone beyond the basics with the git log. For me, this is one of the most useful commands, as it gives you an almost universal ability to query the history of the repository you’re looking at. As I triage git problems at work, git log --graph commands get me up to speed very quickly with what’s gone on and what kind of project I’m dealing with.

Exercises

1) Read the git log man page. Don’t worry about understanding it all, it can read as quite obscure, and is not exactly a page-turner.

Rather, get a feel for what the log command can do - you never know when a bell can ring, and that flag you saw turns out to be the lifesaver to your day.

Squashing Commits

In Part II, I showed the git rebase command, and how it can be used to move, or ‘re-base’ a set of commits.

In this section I will show you how to

You will cover:

- git rebase -i

- git rev-list

By the end of it you will be comfortable with the idea of squashing commits and perhaps consider how you will safely use it in your own projects.

A Worked Example

Type this in to begin the process of working through a real example of squashing and understand what it is:

1 $ mkdir lgthw_squashing

2 $ cd lgthw_squashing

3 $ git init

4 $ touch afile

5 $ git add afile

6 > git commit -m "Initial commit"

7 $ for i in {1..10}

8 > do

9 > echo $i >> afile

10 > git commit -am "commit:${i}"

11 > done

12 $ git status

13 $ git log

You have created a git repository with a history and seen its git log.

Now clone it into a new folder:

13 $ cd ..

14 $ git clone lgthw_squashing lgthw_squashing_clone

15 $ cd lgthw_squashing_clone

16 $ git remote -v

17 $ git log

You can see the remote of this second repository is set to the original one you created above, and the history has ten commits.

We’ve decided that this history is too complicated for us. What we want to do is ‘squash’ these ten commits into a single one that has the number 10 in afile.

The git rebase -i Command

In order to squash a set of commits, we need:

- A reference to the last (latest) commit of the set we want to squash

- A reference to the oldest (first) commit of the set we want to squash

We already have a stable reference to the latest commit, ie where we are at the moment. Can you remember what it is?

We need a reference to the oldest commit in the history. Can you find a way to get it?

There are many ways to do it. The way we’re going to find it is to use the git rev-list command. You should be at the point where you’re comfortable going to read up on a git command, so go off and do that now.

OK? Now, run this command:

17 $ git rev-list --max-parents=0 HEAD

That gets you the original commit, and that’s the method we’re going to use here.

The following command will substitute the output of that command into the git rebase command below:

18 $ git rebase -i $(git rev-list --max-parents=0 HEAD) HEAD

Don’t worry about the substitution part (the $() bit) of the above command if it makes no sense to you, but consider buying the sister book to this ‘Learn Bash the Hard Way’, available at https://leanpub.com/learnbashthehardway which explains this and other bash subtleties.

We have explicitly put HEAD in as the second argument, but technically you don’t need it, as git assumes that HEAD was the second argument if none was given.

Now follow the instructions and try and navigate your way to squashing the commits into a single one, and confirm you did it by using git log.

If you couldn’t do it, then go to ‘Cleanup’ below and start again.

If you still couldn’t figure out what was going on, I’m going to explain a little more here.

The git rebase -i Command Explained

When you run the git rebase -i command, you are confronted with an editable file screen that shows the ten commits on ten lines. Each line looks like this:

pick c30a1bb commit:1

pick c30a1bb commit:2

Then you get a description of the rebase followed by a bunch of comment lines beginning with # that outline various commands that can be run on each commit:

# Rebase 3922435..2816cf1 onto 3922435 (9 commands)

#

# Commands:

# p, pick = use commit

# r, reword = use commit, but edit the commit message

# e, edit = use commit, but stop for amending

# s, squash = use commit, but meld into previous commit

# f, fixup = like "squash", but discard this commit's log message

# x, exec = run command (the rest of the line) using shell

# d, drop = remove commit

Finally, you get some helpful advice at the bottom:

# These lines can be re-ordered; they are executed from top to bottom.

#

# If you remove a line here THAT COMMIT WILL BE LOST.

#

# However, if you remove everything, the rebase will be aborted.

#

# Note that empty commits are commented out

What’s going on?

Well, rather than simply moving a set of commits from one ‘base’ to another, you can now perform a specific action on each commit. The descriptions are fairly self-explanatory, but it’s worth exploring and playing with them a little while you have the chance to do it without breaking anything.

The default is pick (or p for short), which means that the commit will be kept as it was before.

We’re concentrating on the most common task here, which is squashing. There’s a few confusing things about this process that confused me at first glance.

The first point is to note the time order of commits goes from top to bottom. It is fairly obvious here because we numbered the commits, but I often forget

The second point is that you can’t squash all the commits. One must be ‘picked’ in order that the rebase retains a single commit. Otherwise there’s nothing to squash ‘to’!

Other things worth noting here are that the squash and fixup options are the same, except that fixup does not give you the commit message in the following screen.

The drop option just removes the commit and all its changes from the history.

Also, you still need the initial commit, since you need a ‘base’ commit to ‘anchor’ your rebase to. This is why you end up with two commits in your repository after squashing into all but one of the offered commits.

Finally, when you have rebased (and no matter what you do with the commits), you always end up in a ‘detached HEAD’ state. This is because the branch pointer for the local master has not been moved with the rebase. Remember that when you commit, the HEAD and branch pointer are automatically moved for you, but this is a convenience rebase does not provide for you.

Assuming you’ve squashed everything into the ‘10’ commit, then run these commands to make sure you’re on the master branch and that the master branch is pointed to this new squashed commit:

19 $ git branch -f master

20 $ git checkout master

I use this pair of commands quite a lot, as this ‘move the branch pointer’ pattern is a frequent requirement when people get into a mess with git.

You will have seen output like this from the last command:

Switched to branch 'master'

Your branch and 'origin/master' have diverged,

and have 1 and 10 different commits each, respectively.

(use "git pull" to merge the remote branch into yours)

As always with git, when it tells you something it’s really worth reading it carefully. Git is a person of few words, but you should listen to them!

Have a think about this now and try and understand it before moving on.

How Is This Different From git rebase?

You may be wondering what the difference was between this git rebase -i command and the other one we used before without the -i flag.

Underneath it all is the same broader concept behind rebase: take a set of commits, and do things with them. What the interactive (‘-i’) mode gives you is more control over those individual commits.

If you get more advanced you can both move and squash commits at the same time. But ‘one step at a time’ is key when learning git.

Pushing Squashed Commits

You may also be wondering why we did that clone of the origin repository before doing our squashes.

What we’re going to do now is try and push our squashed changes to the origin.

21 $ git push origin

That should have been rejected with an error message.

Read the output carefully.

To /Users/imiell/tmp/lgthw_squashing

! [rejected]        master -> master (non-fast-forward)

error: failed to push some refs to '/Users/imiell/tmp/lgthw_squashing'

hint: Updates were rejected because the tip of your current branch is behind

hint: its remote counterpart. Integrate the remote changes (e.g.

hint: 'git pull ...') before pushing again.

hint: See the 'Note about fast-forwards' in 'git push --help' for details.

If you follow the instructions by issuing a git pull and merging the result:

22 $ git pull

And then repeat your git push origin:

23 $ git push origin

You will get a very long and probably quite hard-to-understand error message that tells you that the push was rejected.

If you understand what’s going on then that’s great.

If you didn’t, the next section will explore it in more detail.

What’s key to understand here is that you’ve tried to push a change to a repository that isn’t configured to accept it. If you think about it, it makes sense: we’ve taken a git repository that’s being worked on, cloned it, then tried to force a change on it. If the current user of that repository wasn’t expecting changes, that could be very disruptive!

So how do things get pushed normally to what people call ‘git servers’, if all git repositories are equal?

The answer is that there is a special type of git repository designed for the purpose of being pushed to (rather than being worked on), and that’s called a ‘bare repository’. We cover that next.

At this point you can try a ‘force’ with the -f flag to try and override what is on the remote. But that won’t work either.

22 $ git push -f origin

Why Squash?

Finally, you might be wondering why to squash when it causes all this bother.

Mostly, squashing is avoided when dealing with changes already seen on remote repositories. Rather, it is more commonly used when preparing changes made locally for submission to another repository.

If you do use it to squash and push to a remote repository (most likely a git server repository) then you may need to force (git -f) a push if you have changed the history. If you have permission, it will go through, but you will have changed the history for others, which can cause much pain for them trying to figure out why they are having to merge new changes in to their cloned repositories.

In sum, squashing is dangerous, and think about how it might affect others before doing it!

Cleanup

Remove this section’s work with:

24 $ cd ..

25 $ rm -rf lgthw_squashing*

What You Learned

This was quite a dense section. In it you learned:

- git rebase -i vs git rebase

- The git rev-list command

- What squashing commits is

- Why squashing can cause problems with remotes

- How to move a branch pointer

Exercises

1) Figure out five other ways of getting the initial commit in a repository.

2) Experiment with the exec command on commits in the interactive rebase above.

3) Construct a git repository similar to the above example where ‘dropping’ a commit has a different effect to a ‘fixup’. above.

4) Work out how to move and squash a set of commits at the same time.

Bare Repositories

In the previous section we stumbled on pushing a change to another repository, and I mentioned that what we needed was a ‘bare repository’.

In this section you will cover:

- What bare repositories are

- Force-pushing with git push -f

- The dangers of squashing and force-pushing with multiple users

By the end, you will have a better understanding of how to set up a ‘central’ repository that users can pull and push to, and understand how and why you can get into ‘git merge hell’, and your understanding will help you figure out what to do when it happens.

Setting Up A Bare Repository

In this example, you’re going to repeat the last section’s work, but this time you’re going to be able to push your commits at the end. This will be achieved by having a bare repository that your two imagined users will pull from and push to.

Type this in:

1 $ mkdir lgthw_bare_repo

2 $ cd lgthw_bare_repo

3 $ git init --bare

Did you spot the difference?

This time you created a git repository with the --bare flag. Have a look around the folder you are in now. What’s the main difference you notice between this and a ‘normal’ repository?

One difference I want to point out is in the config file:

4 $ cat config

[core]

repositoryformatversion = 0

filemode = true

bare = true

ignorecase = true

precomposeunicode = true

Diff this with a ‘normal’ git repository to see the difference, and then research what the differences mean.

You can’t do much with this repository directly, as there’s nowhere to put your content. Now type this and think about what you’re doing:

13 $ cd ..

14 $ git clone lgthw_bare_repository lgthw_bare_repo_alice

15 $ git clone lgthw_bare_repository lgthw_bare_repo_bob

You’ve created two clones of the original bare repository. Each of these will represent the work of a separate user working on a project, Alice and Bob.

Now, as ‘user 1’ run the same commands as you did in the previous section:

16 $ cd lgthw_bare_repo_alice

17 $ git remote -v

18 $ touch afile

19 $ git add afile

20 $ git commit -m 'Initial'

21 $ for i in {1..10}

22 > do

23 > echo $i >> afile

24 > git commit -am "commit:${i}"

25 > done

26 $ git log --oneline

27 $ git push

You’ve made 10 commits as Alice and pushed the to the remote (bare) repository ready to be picked up by Bob.

Now type this:

28 $ cd ../lgthw_bare_repo_bob

29 $ git log --oneline

You should be able to explain why that failed. What do you need to do to get Alice’s change in your log?

30 $ git fetch origin

31 $ git merge origin/master

32 $ git log --oneline

OK, now all three repositories are in sync, with the ten commits in all of them.

Alice Squashes The Commits

Now you (as Alice) are going to squash the commits, as you did in the previous section.

33 $ cd ../lgthw_bare_repo_alice

34 $ git rebase -i $(git rev-list --max-parents=0 HEAD) HEAD

Now squash the commits by making the rebase specification look like this:

p 283e82d commit:1

f 2587d08 commit:2

f 7589012 commit:3

f 558617f commit:4

f cbdca8d commit:5

f 4ec4603 commit:6

f f91bf0c commit:7

f afd7ef4 commit:8

f e23245c commit:9

f b8d1b74 commit:10

And edit the final commit message so it looks like this:

commits:1-10

Now that is done you need to move the master branch pointer to the squashed commit:

33 $ git branch -f master

34 $ git checkout master

Alice Squashes The Squashed Commits

Now what will happen if Alice pushes? Think about it, then guess:

35 $ git push origin master

Yup, it failed. Read the output carefully now, as we are not going to follow its advice:

To /Users/imiell/tmp/lgthw_bare_repo

! [rejected]        master -> master (non-fast-forward)

error: failed to push some refs to '/Users/imiell/tmp/lgthw_bare_repo'

hint: Updates were rejected because the tip of your current branch is behind

hint: its remote counterpart. Integrate the remote changes (e.g.

hint: 'git pull ...') before pushing again.

hint: See the 'Note about fast-forwards' in 'git push --help' for details.

The key phrase above is this one:

Updates were rejected because the tip of your current branch is behind its remote counterpart

But that’s what we wanted! We wanted to be ‘behind’ because we’ve squashed. We don’t want to git pull as it advises, because we want this ‘history’ to override the previous ‘history’ still stored on the remote.

How do you perform this override? Like this:

36 $ git push -f origin master

The -f flag ‘forces’ the push onto the remote repository. This works now because the bare repository is configured to accept it.

This is great, right? Well, if you are Bob, not so much…

Bob Pulls The Changed History

Now Bob comes into work and want to get going with his tasks on this repository. First he wants to get in sync with the remote master branch.

37 $ cd ../lgthw_bare_repo_bob

38 $ git status

Bob’s a typical user, and hasn’t read this book, so he just does a git pull to pick up any recent changes from the remote.

37 $ git pull

Bob gets asked by git to do a merge. That’s strange, thinks Bob, but he doesn’t really understand what’s going on and goes ahead and merges.

You should do as Bob would, and accept the merge as is. Done that? OK.

So where are we now?

38 $ git log

Hmmm. It’s a complete mess. We have all the commits we originally had, plus the squashed one that Alice pushed.

Bob doesn’t notice that (he really should read this book, shouldn’t he?) and pushes the resulting merge up so ‘everything is in sync’.

39 $ git push

He thinks that all looks OK now, and carries on with his work, safe in the knowledge that all is in sync now.

40 $ cd ../lgthw_bare_repo_alice

41 $ git pull

Looks good to Alice - just a fast-forward. That’s healthy, right?

42 $ git log

Oh dear. Alice is now wondering what happened to her lovely squashed changes.

Git Pull And Merge Hell

This is a simplified version of the pain that can arise when history of a branch gets out of sync. Similar things can happen when rebasing changes between point in the repository, or cherry-picking.

If you find yourself doing merge after merge and it seems like you’re in a mess take a step back and think about what may have happened between your repostory and the remote.

Understanding roughly why this happens is going to put you ahead of 95% of git users, and can make you very popular with your colleagues when they are stuck.

Squash On A Branch, And Locally

If there’s a moral to all this, it is that you should squash on a branch, and only locally if possible.

It’s yet another reason to take advantage of git’s cheap cost of branching.

If there’s a second moral, it’s to avoid force-pushing wherever other users are involved. Many git products have options to forbid, or only allow certain privileged users to force-push changes to repositories for this reason.

Cleanup

To clean up:

43 $ cd ..

44 $ rm -rf lgthw_bare_repository*

What You Learned

In this section you learned:

- What a ‘bare’ repository is

- How to set up a ‘bare’ repository

- How to ‘force-push’ a change with the -f flag

- What ‘git pull and merge hell’ is, and how to avoid it

Exercises

1) Set up a bare repository on a server, and research how to clone from it remotely. I suggest using the ssh method, as it’s the simplest to set up.

Cherry-Picking and Three-Way Merges

You may or may not remember that we covered the git cherry-pick command back in Part II. At the time I skated over some detail that I think you’re ready for now.

In this section you will cover:

- The git cherry-pick command (again)

- Why cherry-pick might not do what you expect

- The git diff-tree and git apply commands

- What a ‘three-way merge’ is

By the end, you’ll understand the cherry-pick command much more clearly, and why and how you can get more surgical with your application of diffs from individual commits.

A Simple Branched Repository

This should be fairly routine by now. You’re going to set up a git repository with two branches and some simple changes in them. Once that’s done you’re going to try to cherry-pick a change from the abranch branch and apply it to the master branch.

1 $ mkdir lgthw_patch_and_apply

2 $ cd lgthw_patch_and_apply

3 $ git init

4 $ touch afile

5 $ git add afile

6 $ git commit -m 'afile added'

7 $ echo First change, on master >> afile

8 $ git commit -am 'First change, on master added'

9 $ git branch abranch

10 $ echo Second change, on master >> afile

11 $ git commit -am 'Second change, on master added'

12 $ git checkout abranch

13 $ echo First change, on abranch >> afile

14 $ git commit -am 'First change, on abranch added'

15 $ echo Second change, on abranch >> afile

16 $ echo New file, on abranch >> newfile

17 $ git add newfile

18 $ git commit -am 'Second change, on abranch added'

19 $ git tag abranchtag

20 $ git checkout master

Now you’ve typed that in, use git log to have a look at the repository you’ve created and all its changes. Make sure you look at it with the --all, --graph, and --decorate flags, and then check the changes at each commit with the --patch flag.

It’s important to get a grip on the repository you are looking at before you try and perform surgery on it. This is what we’re about to do as we try and cherry pick a change. Obviously this repository is simplified, but practising this will help you as you go.

A Simple Cherry Pick

Before you type it in, have a think about what you expect the following cherry-pick command does. You may be surprised by what happens, and the more surprised you are, the more likely you are to learn something useful!

First, have a look at the tag abranchtag to see what you’re going to cherry-pick:

21 $ git show abranchtag

You should see two changes: the addition of the ‘Second change, on abranch’ line and a new file called newfile.

Now you’re going to cherry-pick the specific tag referenced by abranchtag. Write down what you think will happen, and then type in:

22 $ git cherry-pick abranchtag

Did that do what you expected?

Have a look at the afile file. Where did the First change, on abranch line come from? That wasn’t in the git show output!

You might be smarter than me, but I was not expecting to see that in there. I was expecting a conflict, but not the First change, on abranch within it. I also expected to see the file newfile, because that file was added on that commit.

In other words, I expected it to just try and apply the contents of git show, and not introduce anything else to the mix.

What it did do was:

- Tried to apply the change in the abranchtag commit

- Saw that there was a conflict, and so

- Performed a three-way merge

We will cover three-way merges next, before showing how to apply only the changes in the abranchtag commit as I originally intended.

‘Falling Back’ to a Three-Way Merge

Let’s picture what the commit history looked like before we did the cherry-pick.

* 93c07a5 (tag: abranchtag, abranch) Second change, on abranch added

* 7fc9dcc First change, on abranch added

| * 2348f4c (HEAD -> master) Second change, on master added

|/

* 9c3f1af First change, on master added

* 9c01ac0 afile added

The HEAD is pointed at the master branch. The abranch branch is pointed at the same commit as the abranchtag tag.

You now apply the cherry-pick of the abranchtag commit (which has an ID of 93c07a5 here, but will be different for you) to the master branch (which is at 2348f4c here.

What git does is go to the first common ancestor (which here is 9c3f1af First change, on master added and see what has changed between the abranchtag commit you want to apply, and the master branch you are applying to. Conflicts are found, so git gives you all the changes on both branches from the first common ancestor and asks you to sort it out.

In other words, git says: “I can’t apply that individual commit here without any conflicts, so I’m going to apply all the diffs between these two points, and let you figure out what’s going on.”

Hence this is called a ‘three-way merge’. The merge compares the changes between the source (abranchtag), the target (the master branch) and the first common ancestor (9c3f1af above) and .

Note that it only does this if there’s a conflict. If there was no conflict in the change you cherry-pick, then the three way merge is not invoked. If you don’t believe me, then start over, but do not run the git commit -am 'Second change, on abranch added' line and see what happens.

Generate Patch

So far you’ve seen that cherry-pick didn’t do what you wanted, and you’ve seen why. Now you’re going to use a different technique to achieve what you want.

First, revert the cherry-pick you started:

23 $ git cherry-pick --abort

Now you’re going to do two things:

- Create a ‘patch’ file that has the diff contained in the abranchtag commit within it

- Apply that diff to the master branch

Patch files were originally created to pass changes round by email. The git format-patch command does this by default, creating a patch ready for email submission.

Few people do that these days, and we’re not emailing this change, so you’re going to use the git diff-tree command.

To create a the patch file, run this:

24 $ git diff-tree -p abranchtag > abranchtag.patch

Now take a look at the contents of that file:

25 $ cat abranchtag.patch

9c9b211ae7a6ba9012b1700c9c2f83d2070ef58c

diff --git a/afile b/afile

index f106a1d..c538437 100644

--- a/afile

+++ b/afile

@@ -1,2 +1,3 @@

First change, on master

First change, on abranch

+Second change, on abranch

diff --git a/newfile b/newfile

new file mode 100644

index 0000000..a245469

--- /dev/null

+++ b/newfile

@@ -0,0 +1 @@

+New file, on abranch

As usual, the identifiers will differ, but with a little patience you will be able to see the changes. Try and figure out what the +, -, and @ symbols in the above output mean. To find out in more detail, the ‘unified’ diff format used above are documented at:

https://www.gnu.org/software/diffutils/manual/html_node/Detailed-Unified.html#Detailed-Unified

Apply Patch

Now you have the patch for that commit (and only that commit), you can apply it to the master branch:

26 $ cat abranchtag.patch | git apply

…but that fails!

It fails because there was a conflict. Take a moment to try and work out why.

Git could not apply the diff because the addition of the Second change, on abranch line to the afile has a line above it (First change, on abranch) which is just not present in the same file on the master branch. So git doesn’t know what to do, and gives up trying to apply the patch.

If you want to check what’s going on at this point, run some git status and git log commands to check that the master branch is still in a good state. You will see that no changes were made to the repository.

To apply the diffs that can be applied, run the same command with the --reject flag:

27 $ cat abranchtag.patch | git apply --reject

This time, changes were applied. The conflict we saw before was put in a .rej file. The .rej file is created with the name of the file the conflict relates to and the .rej appended. In this case, the file created was afile.rej.

Take a look at it:

28 $ cat afile.rej

diff a/afile b/afile (rejected hunks)

@@ -1,2 +1,3 @@

First change, on master

First change, on abranch

+Second change, on abranch

At this point, you have a couple of choices about what to do. You can try to adjust the .rej file and re-apply it using git apply, or just inspect the conflict described in the diff and apply the change directly to the file.

The second approach is the one I invariably take, as fiddling with patch files directly is not for the faint-hearted!

You can finish the job yourself: apply the change manually, and then commit the changes and inspect your repository to understand what you’ve done.

Cleanup

To clean up what you just did:

29 $ cd ..

30 $ rm -rf lgthw_patch_and_apply

What You Learned

This section was quite dense, and covered a lot of ground. However, in very simple form you have covered here some quite sophisticated techniques that might come in very handy at some point when managing a complex git repository.

You covered:

- What git cherry-pick actually does

- What a ‘three way merge’ is

- How to generate and apply a patch to a branch

If you can get these techniques and ideas into muscle and working memory, then you can look like a magician at the keyboard who can bend git to her will!

Exercises

1) Read the git diff-tree and git format-patch man pages and try and follow them. Look up any terms you’re not sure about.

Git Hooks

Git hooks allow you to control what the git repository does when certain actions are performed. They’re called ‘hooks’ because they allow you to ‘hook’ a script at a specific point in the git workflow.

In this section you will cover:

- What git hooks are

- Pre-commit hooks

- Pre-receive hooks

- The git cat-file command

By the end, you should be comfortable with what git hooks are, and able to use them in your own projects.

Create Repositories

To understand git hooks properly, you’re going to create a ‘bare’ repository with nothing in it, and then clone from that ‘bare’ repo. We looked at bare repositories earlier in this (advanced) part.

1 $ mkdir lgthw_hooks

2 $ cd lgthw_hooks

3 $ mkdir git_origin

4 $ cd git_origin

5 $ git init --bare

6 $ cd ..

7 $ git clone git_origin git_clone

Now you have two repositories: git_origin, which is the bare repository you will push to, and git_clone, which is the repository you will work in. You can think of them as part of a client-server git workflow where users treat the git_origin folder as the server, and clones as the client.

Next, add some content to the repository, and push it:

8 $ echo 'first commit' > file1

9 $ git add file1

10 $ git commit -m 'adding file1'

11 $ git push

Nothing surprising should have happened there. The content was added, committed and pushed to the origin repo.

Adding a ‘pre-commit’ Hook

Now imagine that you’ve set a rule for yourself that you shouldn’t work at weekends. To try and enforce this you can use a git hook in your clone.

Add a second change, and take a look at the .git/hooks folder:

12 $ echo 'second change in clone' >> file1

13 $ ls .git/hooks

In the .git/hooks folder are various examples of scripts that can be run at various points in the git content lifecycle. If you want to, you can take a look at them now to see what they might do, but this can be a bit bewildering.

What you’re going to do now is create a script that is run before any commit is accepted into your local git repository:

14 $ cat > .git/hooks/pre-commit << EOF

15 > echo NO WORKING AT WEEKENDS!

16 > exit 1

17 > EOF

18 $ chmod +x .git/hooks/pre-commit

What you have done is create a pre-commit script in the hooks folder of the repository’s local .git folder, and made it executable. All the script does is print the message about not working at weekends, and exits with a code of 1, which is a generic error code in a shell script (exit 0 would mean ‘OK’).

Now see what happens when you try to commit:

19 $ git commit -am 'Second change'

You should have seen that the commit did not work. If you’re still not sure whether it got in, run a log command and check that the diff is still there:

20 $ git log

21 $ git diff

This should confirm that no commit has taken place.

To show a reverse example that lets the commit through, replace the script with this content:

22 $ cat > .git/hooks/pre-commit << EOF

23 > echo OK

24 > exit 0

25 > EOF

This time you’ve added an ‘OK’ message, and exited with a 0 (success) code rather than a 1 for error.

Now your commit should work, and you should see an ‘OK’ message as you commit.

26 $ git commit -am 'Second change'

A More Sophisticated Example

The above pre-commit scripts were fairly limited in their usefulness, but just to give a flavour of what’s possible, we’re going to give an example that is able to choose whether to allow or reject a commit based on its content.

Imagine you’ve decided not to allow any mention of politics in your code. The following hook will reject any mention of ‘politics’ (or any word beginning with ‘politic’).

27 $ echo 'a political comment' >> file1

28 $ cat > .git/hooks/pre-commit << EOF

29 $ if grep -rni politic *

30 > then

31 > echo 'no politics allowed!'

32 > exit 1

33 > fi

34 > echo OK

35 > exit 0

36 > EOF

37 $ git commit -am 'Political comment'

Again, the commit should have been rejected. If you change the content to something else that doesn’t mention politics, it will commit and push just fine.

38 $ echo 'a boring comment' >> file1

39 $ git commit -am 'Boring comment'

40 $ git push

Even more sophisticated scripts are possible, but require a deeper knowledge of bash (or other scripting languages), which is out of scope. We will, however, look at one much more realistic example in last section of this chapter.

Are Hooks Part of Git Content?

A question you may be asking yourself at this point is whether the hooks are part of the code or not. You won’t have seen any mention of the hooks in your commits, so does it move with the repository as you commit and push or not?

An easy way to check is to look at the remote bare repository directly.

41 $ cd ../git_origin

42 $ ls hooks

Examining the output of the above will show that the pre-commit script is not present on the bare origin remote. So it does not move with the code, nor is it considered part of the code committed.

This presents us with a problem if we are working in a team. If the whole team decides that they want no mention of politics in their commits, then they will have to remember to add the hook to their local clone. This isn’t very practical.

But if we (by convention) have a single origin repository, then we can prevent commits being pushed to it by implementing a pre-receive hook. These are a little more complex to implement, but arguably more useful as they can enforce rules per team on a canonical repository.

The pre-commit hook we saw before is an example of a ‘client-side hook’, that sits on the local repository. Next we’ll look at an example of a ‘server-side hook’ that is called when changes are ‘received’ from another git repository.

Pre-Receive Hooks

First type this out, and then I’ll explain what it’s doing. As best you can, try and work out what it’s doing as you go, but don’t worry if you can’t figure it out.

43 $ cat > hooks/pre-receive << 'EOF'

44 > #!/bin/bash

45 > read _oldrev newrev _branch

46 > git cat-file -p $newrev | grep '[A-Z][A-Z]*-[0-9][0-9]*'

47 > EOF

This time you created a pre-receive script, which will be run when anything is pushed to this repository. These pre-receive scripts work in a different way to the pre-commit hook scripts. Whereas the pre-commit script allowed you to grep the content that was being committed, pre-receive scripts do not. This is because the commit has been ‘packaged up’ by git, and the contents of the commit are delivered up as that packaged content.

The read command in the above code is the key one to understand. It reads three variables: _oldrev, newrev, and _branch from standard input (ie the data that ‘coming in’ to the script’. The contents of these variables will match, respectively: the previous git revision reference this commit refers to; the new git revision reference this commit refers to; and the branch the commit is on. Git arranges that these references are given to the pre-receive script on standard input so that action can be taken accordingly.

Then you use the (previously unseen in this book) git cat-file command to output details of the latest commit value stored in the newrev variable. The output of this latest commit is run through a grep command that looks for a specific string format in the commit message. If the grep finds a match, then it returns no error and all is ok. If it doesn’t find a match, then grep returns an error, as does the script.

Make the script executable:

48 $ chmod +x hooks/pre-receive

Then make a new commit and try to push it:

49 $ cd ../git_clone

50 $ echo 'another change' >> file1

51 $ git commit -am 'no mention of ticket id'

52 $ git push

That should have failed, which is what you wanted. The reason you wanted it to fail is buried in the grep you typed in:

grep '[A-Z][A-Z]*-[0-9][0-9]*'

This grep only returns successfully if it matches a string that matches the format of a JIRA ticket ID (eg PROJ-123). The end effect is to enforce that the last commit being pushed must have a reference to such a ticket ID for it to be accepted. You might want such a policy to ensure that every set of commits can be traced back to a ticket ID.

Cleanup

To clean up what you just did:

53 $ cd ../..

54 $ rm -rf lgthw_hooks

What You Learned

We’ve only scratched the surface of what commit hooks can do, and their subtleties and complexities. But you should now be able to:

- Know what a git hook is

- Understand the difference between a client-side and server-side hook

- Implement your own git hooks

Exercises

1) Research the git manuals to see where the other hooks are called in the git lifecycle.

2) Implement a script for each stage of the lifecycle and think of a situation where you might use that in your code workflow.

3) Research how GitHub uses git hooks, and think about what extra features they’ve added to the core git hooks, and why.

Summary

In this final section you’ve extended your knowledge of git beyond what’s required to do your day job well. While it may be considered optional, this section has enabled you to do more advanced tasks like quickly figuring out a messy git history that someone brings to you, squashing commits (and changing history), applying individual commits more surgically than using cherry-pick, and serving your own repositories by setting up bare repositories.

More importantly, we’ve taken the training wheels off and started to let you take control of your learning. You should be at the point now where you can start to research and understand git challenges yourself.

Where you go from here is up to you and it could be in a number of directions. You might want to look more into git internals, get quicker at diagnosing your colleagues’ problems, restructure your git workflow, set up your own git server and serve your own repositories privately, or even start looking and contributing to the core git source.

Whatever you do, make sure that you let me know how you get on! Contact details are at the end of this book.

Finished!

Well done! You’ve finished the course.

The course’s aims were to give you an understanding of key concepts of git so that you:

- Understand what key git concepts

- Able to develop your understanding of git further

- Have a practical facility with git that makes you able to diagnose and fix others’ problems

If I had to put it in a sentence, I’d say that this course should give you the ability to never be surprised or helpless in the face of any git situation you come across.

I hope that’s what you got from the course!

I welcome any feedback to ian@mail.meirionconsulting.com or on Twitter: @ianmiell

Note also that I offer training in Git, Bash, and Docker. Just get in touch if you want to know more.