Results 1 to 3 of 3

Thread: In the beginning there was the command line...

  1. #1
    Forum Administrator Power Poster Lyle@Spry's Avatar
    Join Date
    May 2005

    Default In the beginning there was the command line...

    There seems to be a common misconception that "SSH" is some magical system administration tool that many of our clients have never used, and would like tutorials for how to use it to:
    • Install software
    • Back up their sites
    • Monitor usage
    • Set up multiple email accounts
    • Upload website files
    • ...and countless other system administration tasks

    I'd like to take some time with this post to explain the difference between SSH (the protocol) and the Linux (*nix) command line.

    SSH is (to put it briefly) a protocol and software to access a *nix computers command line interface. Much like the "Command Prompt" application in Microsoft Windows. When you open the command prompt tool on Windows, it opens a window showing the MSDOS prompt. For Mac users, the Applications->Utilities->Terminal does the same (although not MSDOS, it shows the bash or tcsh command line shell.)

    When you use a program like Putty to SSH to your VPS, you are essentially opening a connection to the server and starting a shell. (most Linux distributions install bash as the default shell, though there are many alternatives you can choose from) I think this page explains it pretty well:

    What is PuTTY?
    PuTTY is a client program for the SSH, Telnet and Rlogin network protocols.
    These protocols are all used to run a remote session on a computer, over a network. PuTTY implements the client end of that session: the end at which the session is displayed, rather than the end at which it runs.
    In really simple terms: you run PuTTY on a Windows machine, and tell it to connect to (for example) a Unix machine. PuTTY opens a window. Then, anything you type into that window is sent straight to the Unix machine, and everything the Unix machine sends back is displayed in the window. So you can work on the Unix machine as if you were sitting at its console, while actually sitting somewhere else.
    So in essence, SSH merely connects you to the command line of your VPS. If you are not familiar with command line administration of a Linux system, I must warn you that it is not easy, and the learning curve is quite steep. (Hence the proliferation of web based control panels like Webmin/cPanel/Plesk/ServerCP/DirectAdmin/Ensim which aim to reduce the need to use the command line at all for common tasks...) I've personally been using Linux for 13 years, and still finding new ways of doing things from command line (although it has long since become my preferred interface for managing *nix systems.)

    With that said, here are some examples of things you can do using the command line.

    Navigating around the filesystem:
    Just like in MSDOS, the cd command changes the current working directory. UNlike MSDOS, directories use the / character in place of \. (Mac users rejoice, no difference, Mac also uses /. Don't ask why *nix does it different, it was around first, MSDOS/Windows decided to be different.) So the command cd /var/www/html moves you to /var/www/html on your server.

    Viewing the contents of a directory/folder:
    The ls command prints out the contents of the current working directory. So if you are following along, typing ls while inside the /var/www/html directory shows the files in that directory. You can also view the contents of directories you are not currently in by giving the full path to the directory. ls /tmp would show the list of files in the /tmp directory on your server.

    Viewing the contents of a file:
    There are several commands available for this, all with slightly different uses. The cat command simply writes the contents of the file to the screen all at once. This is useful for small files which don't take up the entire screen. cat /etc/groups prints out the contents of the file /etc/groups. If you want to view a larger file, a pager is more useful (pagers display at most a page of text at a time, allowing you to navigate through the file in steps.). There are two programs commonly installed on Linux systems to do this, more and less (Yes, that is a play on words, you'll find that Linux developers have a funny sense of humor.) The more command displays the contents of a text file, and allows you to move forward through the file one line at a time by pressing <Enter> or <Down Arrow>, or a page at a time by pressing <Space> or <Page Down>. more /home/user/public_html/somelongpage.html The less command does a bit more than more (Funny, eh?), it also allows you to scroll up if you passed what you wanted to read by pressing <Up Arrow> or <Page Up>. less /home/user/public_html/somelongpage.html

    Renaming a file:
    This is essentially the same as moving a file to another directory, which is accomplished with the mv command. To move /home/user/somefile to the /tmp directory: mv /home/user/somefile /tmp, to rename /home/user/somefile to /home/user/someotherfile: mv /home/user/somefile /home/user/someotherfile

    Deleting a file:
    This is done with the rm command. rm /home/user/somefile This command can also delete multiple files at one time, just give all the filenames on the command line: rm /home/user/somefile /home/user/someotherfile To remove a directory, give the directory name: rm /home/user/tmpdir, but only if the directory is empty (contains no files or directories). To delete a directory and the files and directories below it we must use a couple command line options. In this case, -rf. Command line options usually start with a - character, and are immediately followed by a single letter (or multiple letters signifying multiple different options.) The -r option means to recurse into the directory, deleting files it finds in the directory, and the -f option means to force the operation, deleting the directory even if files exist inside the directory. Most Linux commands accept a multitude of options to alter their default behavior. (For instance, the ls command accepts a multitude of options to change the order in which files listed are displayed.)

    Running a script/program:
    Up until now, we've been running commands (ls, mv, rm) by simply typing their name (and any options/arguments) and hitting <Enter>. This is because these commands/programs are installed in standard locations. When you execute a command using the shell, the shell looks through a list of directories for a file named like the command and then runs it with your selected options. This list of directories is called your PATH environment variable. You can see the current list of directories in your PATH by typing echo $PATH or env|grep PATH. You may already be familiar with the idea of PATH, as it is also used by MSDOS/Windows/Mac. If you want to run a program/script that is NOT installed in one of the directories in your PATH, you can either 1) modify your PATH environment variable, or 2) Give the full path to the program/script on the command line.

    For example, cPanel installs many useful scripts to fix various common problems in the /scripts directory. If the user quotas get messed up for some reason, you would run the /scripts/fixquotas script. If you just typed fixquotas (Even if you are in the /scripts directory) you would most likely receive the error message:
    bash: fixquotas: command not found
    To run the script, either give the full path: /scripts/fixquotas or modify your PATH to include /scripts: export PATH=$PATH:/scripts. (This command appends /scripts to the end of the current $PATH variable and resets the PATH variable. It only lasts until you log out.) Along the same lines, you may have seen reference to running a script in the current directory by prepending ./. This works because [b].[/] is a shortcut meaning the current working directory. So if you cd /scripts first, you can also run the fixquotas script by typing ./fixquotas. Internally the shell replaces . with /scripts so you're really running /scripts/fixquotas.

    This post has ballooned to past the forum post length limit. More to come in future posts in this same thread. Please post any comments or requests here.

  2. #2
    Forum Administrator Power Poster Lyle@Spry's Avatar
    Join Date
    May 2005

    Default Installing software...

    To continue in the original purpose of this thread, we'll move on to how to install software from the command line. We'll eschew package managers like RPM or DEB for source compilation since it applies to all linux distributions equally. If your linux distribution offers a binary package (rpm, deb) for the software you wish to install, by all means use that instead. Compiling from source allows much greater control, and package maintainers sometimes take several days/weeks/years to release an updated package after the developer releases a new version of the source. So if you want the very latest version of a software package, source compilation is often the only way.

    Before I get too far, note that MOST *nix software can be installed following these commands, but definitely NOT ALL. Be sure to follow the developers documentation for detailed instructions for how to install their software. (Most *nix programs come with a README or INSTALL file, they should be considered the ultimate authority on installing that particular piece of software, regardless of what is written here.)

    In this example, we will be installing GNU hello, which was written
    as a model for all of the GNU coding standards and other recommended practices.
    First up, we'll need to get the source onto the server. You could simply download hello-2.3.tar.gz to your desktop computer and then upload it to your server, but why bother with two trips? SSH to your server and run this:
    curl -O
    or, if you don't have curl installed:
    Both of which are command line tools to download a given url. (wget even goes as far as to call itself a non-interactive web browser!)

    You should now have the file hello-2.3.tar.gz in your current working directory on the server.

    Next, we'll need to extract this tar.gz archive (it's like a zip or rar file) commonly called a tarball.
    tar xzf hello-2.3.tar.gz
    Now you should have a directory under your current working directory named hello-2.3. Go inside it with
    cd hello-2.3
    Now, READ the provided documentation! There are both a README and INSATLL files in this directory.
    Now that you've read the included documentation, you should know that our next step is to configure the source for compilation on our system. The configure script in the current directory is a product of the GNU autotools toolchain. Software developers use these tools to create configure scripts like this one to save time and headaches. Similar to installer products on other computer architectures like MSI or InstallShield, Macrovision's InstallAnywhere, Wise...

    This configure script accepts many command line options to tune the way the software should be installed. To see all the supported options, run it with the --help option:
    ./configure --help
    (See previous post for an explanation of the "./")

    We'll just install with the default options, so run:
    A load of text will fly by:
    /tmp/hello-2.3$ ./configure
    checking for a BSD-compatible install... /usr/bin/install -c
    checking whether build environment is sane... yes
    config.status: creating Makefile
    config.status: creating contrib/Makefile
    config.status: creating doc/Makefile
    config.status: creating gnulib/lib/Makefile
    config.status: creating man/Makefile
    config.status: creating po/
    config.status: creating src/Makefile
    config.status: creating tests/Makefile
    config.status: creating config.h
    config.status: executing depfiles commands
    config.status: executing po-directories commands
    config.status: creating po/POTFILES
    config.status: creating po/Makefile
    Notice how it is checking for numerous things, and then creating several files. It is checking for the software and libraries required by the software to run, and creating Makefiles. Makefiles are part of the autotools toolchain, and specifically say how the software should be built and installed.

    Now we build the software according to the rules in the created Makefile. Simply type make. A bunch more cryptic text flies by:
    /tmp/hello-2.3$ make
    make all-recursive
    make[1]: Entering directory `/opt/admin/tmp/hello-2.3'
    make[2]: Leaving directory `/opt/admin/tmp/hello-2.3'
    make[1]: Leaving directory `/opt/admin/tmp/hello-2.3'
    At this point, the software is built, but it isn't installed in a standard location (binaries in a directory on a standard PATH...). So to do that, we run make install. You may have noticed that my prompt character is a '$', which on most systems indicates I am running as a normal (non-privileged/non-root) user. To install the built software into standard locations requires root privileges, so I'll prepend make install with sudo. If you are following along as root, leave that off and just run make install:
    /tmp/hello-2.3$ sudo make install
    Making install in contrib
    make[1]: Entering directory `/opt/admin/tmp/hello-2.3/contrib'
    make[2]: Leaving directory `/opt/admin/tmp/hello-2.3'
    make[1]: Leaving directory `/opt/admin/tmp/hello-2.3'
    Congratulations, GNU Hello is now installed!
    $ hello
    Hello, world!
    Now, you can either delete the hello-2.3 directory and hello-2.3.tar.gz files, but I like to keep the directory around for a little while at least. Most GNU autotools installable software also supports the make uninstall procedure to remove the installation should you decide you don't want it anymore. (You have to keep the directory because the Makefile(s) are what contain the list of files to remove on uninstall. make uninstall run from outside of the source directory will result in the error "make: *** No rule to make target `uninstall'. Stop."

    That's all, if you have any questions or comments about this, please feel free to reply here.

  3. #3
    Forum Administrator Power Poster Lyle@Spry's Avatar
    Join Date
    May 2005

    Default Shell Scripting

    So now you've probably had a chance to play with the command line a bit, and still aren't quite convinced of its great power. Just wait, we're getting to the best part!

    Any command you can run from the command line, can also be written to a file and stored for later execution. (For instance, say you've got a string of commands that update files on your website, or a group of directories full of temp files that you delete on a regular basis.)

    To do so, create a text file and put the following on the very topmost line of the file:
    You'll also need to change the permissions on the file to make it "executable".
    chmod a+x filename
    Now you can add lines to this file (below the "shebang" line) and when you run this script, all the commands in it will be run as if typed at the command prompt. (Assuming you are using bash as your login shell.) Some other commonly used shells are:
    • /bin/ash
    • /bin/bsh
    • /bin/ksh
    • /usr/bin/ksh
    • /usr/bin/pdksh
    • /bin/tcsh
    • /bin/csh

    If you prefer to use tcsh for example, your script should then instead start with:


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts