Dec 21 2008
Synchronize Directories with Rsync and SSH
To copy or synchronize data across servers with SSH encryption benefits, run on the target:
rsync -avz -e ssh remoteuser@remotehost:/remote/dir /local/dir/
Dec 21 2008
Linux and Solaris Package Managers
Here’s a package managers basic commands quick reminder/comparative for (in this order)
– Linux Debian/Ubuntu,
– Linux Redhat/Suse and
– Solaris/OpenSolaris.
rpm and dpkg are basic package managers while yum (rpm) and apt-get/aptitude (dpkg) manage dependancies and (online) repositories.
# List ALL installed packages
dpkg --list rpm -qa | grep rpm pkginfo
# Remove package
dpkg --remove package or aptitude remove package or apt-get remove package rpm -e package pkgrm package
# Package installation
dpkg --install package or aptitude install package or apt-get install package rpm -ivh package pkgadd -d package
# Package upgrade
aptitude safe-upgrade package rpm -Uvh package
# Package information
dpkg --status package or aptitude show package or apt-cache show package rpm -q --info rpm pkginfo -x package
# List package files
dpkg -L package rpm -ql package pkgchk -l package
# What package a file belongs to
dpkg --search /usr/bin/dpkg rpm -qf /usr/bin/rpm pkgchk -lp /usr/bin/ls or grep "/usr/bin/ls" /var/sadm/install/contents
# Adding a Repository source
Debian/Ubuntu
/etc/apt/sources.list. Then run aptitude update
Redhat/Fedora/Suse
cd /etc/yum.repos.d Create myown.repo [myown] name=Myown RPM Repository for Red Hat Enterprise Linux baseurl=http://myownrepos/redhat/$release/en/$basearch/ gpgcheck=1 enabled=1 Import the GPG key rpm --import http://myownrepos/packages/RPM-GPG-KEY.txt
Debian advise using aptitude rather than apt-get as it’s supposed to manage dependancies in a better way.
Nov 20 2008
Environment Variables and SSH, How to
Very few environment variables are defined when connecting to a remote host with an authorized SSH key. The env command that will list all environment variables available on the remote server shows this is the case (localhost is the target here).
Sometimes, you’d like to pass extra environment variables onto the host along the SSH command. For instance, to run a program calling libs from an unusual location, the LD_LIBRARY_PATH variable has to be set.
If You Have root Access on the Remote Host
This can be achieved in 2 simple steps:
– Add ‘PermitUserEnvironment yes‘ to the sshd_config file on the remote server and restart the SSH daemon.
PermitUserEnvironment default value is ‘No’ most of the time.
$ ssh localhost 'sudo sshd -T' | grep -i environment
permituserenvironment no
– Set environment variables in .ssh/environment in your home directory on the target host in the variable=value format.
eg LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib
Environment Variable as Part of the SSH Command
If changing the sshd configuration on the remote host is not an option, you can always pass the environment variable along the SSH command. It is less elegant, especially in the previous use case since the path depends on the path libraries have been installed in, on that specific host. The escaping might be tricky, but this solution does the job as well.
$ ssh localhost LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib env
LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib
SHELL=/bin/bash
SSH_CLIENT=127.0.0.1 53816 22
USER=dave
MAIL=/var/mail/dave
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
PWD=/home/dave
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/dave
LOGNAME=dave
SSH_CONNECTION=127.0.0.1 53816 127.0.0.1 22
_=/usr/bin/envNov 15 2008
Compile Twice as Fast with Make
Even though GNU “make” utility retrieves a lot of information off the system it’s running on, many times I have to wait a long time to get binaries compiled. It seems “make” doesn’t always pick the best settings to compile applications. I searched how I could speed up compilation times and here are some interesting results.
build_server$ make
5m7s
I wasn’t getting the best performances out of my Linux Redhat, so I turned off the cpuspeed service. cpuspeed reduces or raises the clock speed basically to save power according to the man page. This can have a significant impact when you need to release the CPU’s full capacity: 12% faster in my case:
build_server$ /etc/init.d/cpuspeed stop
build_server$ make
4m30s
Second setting you can play with is the number of jobs “make” can run simultaneously. This is particularly interesting when compiling on multicore servers, which is most likely the case nowadays. It looks like setting the jobs to the number of cores gives the best results with the -j command option. Number of cores can be obtained with the top or cat /proc/cpuinfo commands. This is another 48% gain, making it 53,7% overall.
build_server$ make -j 4 # number of CPUs
2m22s
CI/CD is spreading fast and IT departments build and compile sometimes many times a day. It is crucial to reduce the time spent in compiling applications. These 2 tricks speed up compile times by over 50% which is pretty awesome. This test was conducted on compiling php on a Redhat distribution.
Nov 07 2008
Additional Swap in File on Linux/Solaris
You need more swap but there’s no more disk space left to create a partition? Here’s a quick and easy fix creating a swap file on the existing filesystem. No reboot needed!
Solaris
[root@solaris]$ mkfile 1024m /export/tmpswap
[root@solaris]$ swap -a /export/tmpswap
[root@solaris]$ swap -l
swapfile dev swaplo blocks free
/dev/dsk/c1t0d0s1 55,65 8 4160824 4160824
/export/tmpswap - 8 2097144 ©
Linux
[root@linux]$ dd if=/dev/zero of=tmpswap bs=1024 count=10240
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 0.528435 seconds, 19.8 MB/s
[root@linux]$ mkswap tmpswap
Setting up swapspace version 1, size = 10481 kB
[root@linux]$ swapon tmpswap
[root@linux]$ swapon -s
Filename Type Size Used Priority
/dev/hda2 partition 409648 0 -1
/tmp/tmpswap file 10232 0 -2
Do not use swap files on a personal computer that is configured to “suspend to disk” since it requires a dedicated partition to save the memory content while going to hibernation. This is not a problem on servers.