Tuesday, December 11, 2012

How to make a Raspberry Pi solar-powered FTP server

http://reviews.cnet.co.uk/desktops/how-to-make-a-raspberry-pi-solar-powered-ftp-server-50009923


You've set up your Raspberry Pi using our easy to follow instructions. You've had a gander at our 25 top fun things to do and now you fancy something a bit more involved. How about making a solar-powered FTP server?
You'll always have instant access to all your digital files, from anywhere with an Internet connection, and it won't cost a penny on your electricity bill.

Ordering the sun bed

We'll be using a simple custom-built £25 Raspberry Pi case, with all the right slots for its outputs, that comes with a small solar panel, a battery case and a micro-USB cable. You'll just need to supply your own NiMH rechargeable batteries.

Point your browser over here -- you'll find all the information you need to place an order via PayPal. The maker, Cottonpickers, has an eBay page as well if you're more comfortable making your purchase that way.

Static IP address

Once it's plopped through your letterbox, slot your Pi into the case and hook it up to power and a monitor. Let's start programming. The first step is to make sure the RPi has a static IP address, as we're going to have to poke a hole through our network firewall to allow incoming FTP requests.
In the RPi desktop, double-click the 'LX Terminal' icon to drop into the Terminal. To setup a static IP address, type the following:
sudo nano /etc/network/interfaces
This file controls the IP addressing for the RPi. All you need to do is scroll down slightly to the 'iface eth0' line and remove 'DHCP' and replace it with 'static'. Now, on the line directly below, enter an IP address for your RPi, along with the subnet mask and gateway. Our example looked like this, but yours will be dependent on your home network:
address 192.168.1.93
netmask 255.255.255.0
gateway 192.168.1.254
Assuming you don't know these already, you can find your IP address and router settings by consulting your router documentation, or by typing the following into a Terminal on the RPi: ifconfig -a. This will list the current IP address, netmask and gateway as configured by the router's DHCP. All you need to do then is enter the IP address and so on into the 'static' section of the file to make sure the RPi will forever boot with that unique IP addess.
After you've entered the details, exit by pressing Ctrl+X to exit, followed by 'Y' to accept the changes, then press Enter a couple of times to get to the command line. Now type:
sudo /etc/init.d/networking stop
sudo /etc/init.d/networking start

This will restart the networking components with the new IP address in place.

VNC

From the Terminal type the following, pressing Enter after each line:
sudo apt-get update
sudo apt-get install vnc-server
vncserver

When the packages have downloaded and installed, follow the instructions on-screen to setup a password, and confirm it, but answer 'No' to the view-only option.
Now that VNC is installed, we need to make sure it loads up as a service every time the RPi reboots. To do this, from the Terminal type:
sudo nano /etc/init.d/tightvncserver, and press enter
In the editor, type the following:
#!/bin/sh
# /etc/init.d/tightvncserver
# Set the VNCUSER variable to the name of the user to start tightvncserver under
VNCUSER='pi'
case "$1" in
   start)
     su $VNCUSER -c '/usr/bin/tightvncserver :1'
     echo "Starting TightVNC server for $VNCUSER "
     ;;
   stop)
     pkill Xtightvnc
     echo "Tightvncserver stopped"
     ;;
   *)
     echo "Usage: /etc/init.d/tightvncserver {start|stop}"
     exit 1
     ;;
esac
exit 0


Now press 'Ctrl+X', then 'Y' to save, followed by 'Enter' a couple of times to get you back into the Terminal. What we need to do now is edit the permissions of the script we've just created so that it's executable, do this by typing in:
sudo chmod 755 /etc/init.d/tightvncserver, and press Enter.
Finally, we need to add it to the start-up scripts, by typing:
update-rc.d tightvncserver defaults, and press Enter.
What you can do now is unplug the RPi from your monitor and locate it somewhere that has easy access to a network cable, and plenty of sunlight (for the solar cells). If you install the likes of TightVNC Viewer you should be able to point the client to '192.168.1.93:1' (or whatever your static IP address is) and have full access to the RPi and the GUI.

VSFTPD

This next part involves setting up the FTP server itself. Again it's not too difficult, and can be configured to your exact preferences later on. From the Terminal, type the following:
sudo apt-get install vsftpd, and press Enter.
Once the VSFTPD (which stands for Very Secure FTP Daemon) packages have downloaded and installed, type:
sudo nano /etc/vsftpd.conf, and press Enter.
This is the configuration file and control for VSFTPD, it allows you to set all sorts of restrictions and policies, so some care is advised. To start with though, we would recommend altering the following lines within the script, either by un-hashing, or typing in 'YES' or 'NO':
Anonymous_enable=NO
Write_enable=YES
Local_enable=YES
Ascii_upload_enable=YES
Ascii_download_enable=YES

The full explanation can be found within the hashed comments in the script itself, but they will suffice for our little project for now. When ready, rave the changes to the script as before.
Finally, reboot the RPi to bring everything you've done so far into effect. To do this, make sure you're in the Terminal, and type:
sudo reboot, and press Enter.

Access external hard drive via FTP

We like to make things easy, so here's our method of accessing an external hard drive attached to one of the RPi's USB ports, via an FTP client.
First, you can hook up an external USB drive to your PC and format it as NTFS, with the volume labelled FTP. When that's complete, plug it in to the RPi and via VNC click 'Yes' to view the drive from within the File Manager.
Make a note of the address of the hard drive -- in our case it came out as /media/FTP. To test FTP access, install an FTP client such as FileZilla and enter the connection details:
192.168.1.93/media/FTP, replacing the IP address with your own static RPi address.
Username: pi, replace this with the username you set VSFTPD with (default is pi).
Password: raspberry, replace this with user's password (if other than pi).
With luck, you should now have access to the external USB hard drive, via FTP, internal to your own network.

External access

The final part of this project involves granting external FTP access to the RPi. This is also the most ambiguous of the steps, as it all depends on the make and model of your router. For example, on the basic BT model I used it's a simple case of selecting FTP from the pre-defined applications, and assigning it to the internal IP address of the Raspberry Pi.
Yours may be completely different, however. That being the case, please consult your router's documentation, or Google the model and see if there's already an FTP external access tutorial out there.

Regular expreesion in Linux Bash and KSH Shells

http://scripting.linuxnix.com/2012/12/regular-expreesion-in-linux-bash-and.html

Many people thinks that RegExp is alian to Bash/KSH Scripting and depends on GREP or SED to use regexp extensively.  But from Version 3 of Bash we can use regular expression with out using grep or sed. This will save us lot of time and reduce number of lines of script we write.

Lets take some example, I want to ask a user to enter "Yes" to do some activity depending on his input. He may end up with entering YES or Yes or YEs or some other form of YES or even just Y or y.

Earlier versions of BASH and KSH we have to depend on grep to put a check on user's different inputs as shown below code.


read option1

echo $option1 | grep -E '[Yy][Ee][Ss]|[Yy]'

if [[ "$?" -eq 0 ]]

then

echo "Do some thing"

else

echo "Do other thing"

fi

output:
Do Some thing

If you see above code we are using 3 commands(echo, grep and if statement) to accomplish our task.

Which will use system resources a bit more when we can accomplish with a just BASH/KSH alone.


From BASH version 3 there is an inbuilt regex operator( =~ ) which will help us to solve this problem



If you are new to regular expressions please click here

Below are the types of regular expressions we have and we will go with each and every regexp with an example.


Basic Regular expressions


^ --Caret symbol, Match beginning of the line/Variable.

$ --Match End of the line/variable.

. --Match Any single character.

* -- Match 0 or more occurrence of previous character(This is bit tricky).

[] – Match Range of characters, just single occurrence.

[a-z] –Match small letters

[A-Z] –Match cap letters


[0-9] – Match numerical.

[^] – Match Negate a sequence of char followed by ^.

\ -- Match Escape character, for matching special chars like ., * etc.


Extended Regular expressions


{n} --Match number of occurrence of a character

{n,m} --Match a character which is repeated n to m times.

{n,} --Match a repeated character which is repeated n or more times.

+ --Match one or more occurrences of previous character.

? – Match 0 or 1 occurrence of previous character.

-- Match Either character

() –match a group of characters


Let us start with an example

I have a variable var1 whose value is "abbbcd acdef 123 acd cda2". And we are going to use same variable for below examples.

Example1: Check var1 start with "a" or not

if [[ "$var1" =~ ^a ]]
then
echo "Yes, it start with a"
else
echo "It does not start with a"
fi

Output:

Yes, it start with a

Note: Dont give quotes around regexp, Your bash/ksh will take care of that.

Example2: Check if var1 ends with 2 or not


if [[ "$var1" =~ 2$ ]]
then
echo "Yes, its end with 2"
else
echo "It does not end with 2"
fi

Output:

Yes, it end with 2

Example3: Check if var1 have string which start with a and ends with d and should contain only 1 letter between them.

if [[ "$var1" =~ a.d ]]
then
echo "Yes, its present"
else
echo "It not there"
fi

Output:

Yes, its present

Example4: Check if var1 have a string which have may contain's 0 or more times of b

if [[ "$var1" =~ b* ]]
then
echo "Yes, its true"
else
echo "It not there"
fi

Output:

Yes, its true.

Note:The output of this script is always true because it we can have var1 with b's and with out b's. We have to depend on + for a meaning full checking like at-least 1 b occurrence.

Example5: Check if var1 contain contains either abc or adc word in it.

if [[ "$var1" =~ a[bd]c ]]
then
echo "Yes, its present"
else
echo "It not there"
fi

Output:
Yes, its present

This will match either abc or adc and gives the output as it present.

Example6: Check if var1 contains abc or bbc or cbc or dbc to zbc

if [[ "$var1" =~ [a-z]bc ]]
then
echo "Yes, its present"
else
echo "It not there"
fi

Output:

Yes, its present

I will leave [A-Z] and [0-9] option to reader it self. Explore your self on this.

Example7: Check if var1 do not contain z in it


if [[ "$var1" =~ [^z] ]]
then
echo "Its not there"
else
echo "z is present"
fi

Output:

Its not there

Example8: Check if a string "123 acd" is present in var1 or not

if [[ "$var1" =~ 123\ acd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi

Note: If you observe regexp, We did not used quotes, your bash/ksh will take care of it.

Output:
Yes, its present.

Example9: Check if var1 contain a string which have 2 b's in between a and cd.

if [[ "$var1" =~ ab{2}cd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi

the output will be "Its not present" because our var1 variable(abbbcd acdef 123 acd cda2) have 3 b's between a nd cd.


Example10: Check if var1 contain a string which have 2 or 3 b's between a and cd.

if [[ "$var1" =~ ab{2,3}cd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi

The output will be "Yes, Its present", because it try to match abbbcd in the string, if one matches.

Example11: Check if var1 contain a string which have 2 or more b's between a and cd.

if [[ "$var1" =~ ab{2,}cd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi

Output:

Yes, its present

Example12: Check if var1 contain 1 or more b's between a and cd

if [[ "$var1" =~ ab+cd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi


Output:

Yes, its present


Example13: Check if var1 contain 0 or 1 b between a and cd

if [[ "$var1" =~ ab?cd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi


Output:

Yes, its present


Example14: Check if var1 contain either acdef or acd in it?


if [[ "$var1" =~ acdef|acd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi

If either one of acdef or acd is there, if loop will be true.

Example15: Check if var1 contain abbbcd or acd in it. We can combain the letters which are common in both the words with () and

if [[ "$var1" =~ a(bbbc|c)d ]]
then
echo "Yes, its present"
else
echo "its not present"
fi


Output:

Yes, its present


if you want to make this more simple you can write the above regexp as below.

if [[ "$var1" =~ a(bbb|)cd ]]
then
echo "Yes, its present"
else
echo "its not present"
fi


Now come back to our old code on getting user input of Yes with out using echo and grep commands..

echo $result | grep -E '[Yy][Ee][Ss]|[Yy]'
if [[ "$?" -eq 0 ]]
then
echo "Do some thing"
else
echo "Do other thing"
fi

The above code can be written as

if [[ "$var1" =~ ([Yy]|([Ee][Ss])) ]]
then
echo "Yes, its present"
else
echo "its not present"
fi

So that user can type Y or y or yes or Yes or YEs or YES or YeS or yEs or yES or yeS. All these things will be matched.

Keep visiting us to get more update on Shell scripting.

Monitoring your server with tmux

http://www.linuxuser.co.uk/news/monitoring-your-server-with-tmux


There are lots of systems and utilities available to monitor your system. Many of these are web-based, or they run as a client-server system.
Unfortunately, there are several instances where the only allowed connection to the server of interest is over SSH. This might be for several reasons, the least of which being security. In these cases, you will likely still want some way of easily monitoring what is going on with your server. Using tmux, you can create a session which will run all of your monitoring software and
keep it running, regardless of whether you lose your connection or not.
This article will cover the basics of creating such a session, which you should be able to tune and tweak to fit your specific requirements. This way, you can simply log in using any available SSH connection and see, in an instant, all of the information that is of interest to you. Also, since you need to log into the system over SSH, you don’t need to worry about the problems of locking down other software, such as a web server.
server monitor
You can monitor a lot of processes and logs with tmux

Resources

tmux
Launchpad

Step by Step

Step 01

Getting tmux
Tmux originated as part of the OpenBSD system. It should be available in most distributions. For example, you can get it in Ubuntu with ‘sudo apt-get install tmux’. If you need the latest and greatest features, you can download the source code from SourceForge.

Step 02

Building tmux
The build system uses the usual ‘./configure; make; make install’ steps to build tmux. The reason you may want to build your own is that many distributions are behind one or more versions on the software provided by their respective repositories.

Step 03

Starting tmux
Starting tmux is as simple as typing ‘tmux’ and hitting Enter. Your console will clear for a split second, and then you will be presented with a Bash prompt again, along with a status bar located at the bottom of your screen. This status bar will contain information about your current tmux session.

Step 04

Starting top – CPU sorted
One of the things you will be interested in monitoring is which processes are using up the most CPU cycles on your server. A good tool for this is ‘top’. The default when you first start it is to sort processes based on CPU usage, so that is fine.

Step 05

Getting a new window
Here we come to one of the features of tmux; we need to create a new window in this tmux session. There are two ways to handle this: first, you can use the shortcut ‘C-b c’, or you can enter the complete command ‘new-window’. To enter commands, you need to enter ‘C-b :’ and then the command. This will put your current window into the background and open a new window in the foreground.

Step 06

Starting top – memory sorted
With a new window, you can start a new instance of top, sorting it on some other criteria. One of interest to most system administrators is which processes are using up memory. To sort the processes in this way, you will need to enter ‘M’. This may vary for other versions of top, so always check your version’s man page. You might also want to change the refresh rate by entering ‘d’ and setting the number seconds between each display.

Steo 07

Navigating windows
Now that you have multiple windows, you need to be able to navigate between them. The simplest way is to use the shortcut navigation keys. To move to a specific window, you can use ‘C-b’ and then the window number. Remember that window numbering starts at 0. If you simply want to move to the next or previous window, use ‘C-b n’ or ‘C-b p’.

Step 08

Creating new panes
The next great feature of tmux is the ability to break up windows into panes. This lets you have multiple programs running in the same window. To split the current pane horizontally, use ‘C-b %’ to get two panes, left and right. If you wish to split the current pane vertically, you would use ‘C-b “’.

Step 09

Navigating panes
Once you end up with multiple panes, you need to be able to navigate them. To move to the next pane in the current window, you would use the shortcut ‘C-b o’.
You can also rearrange panes within a window. To swap the current pane with the previous pane, use the ‘C-b {’ keyboard shortcut. To do so with the next pane, use ‘C-b }’.

Step 10

Using tail
Now that you have tmux essentials under your belt, it’s time to add some systems monitoring. You’ll want to monitor system logs, and you can do so in multiple panes, giving you an overall view. For example, navigate to an empty pane and enter:
tail -f /var/log/syslog
…in order to get a continually updating view of system messages.

Step 11

Following dmesg
Kernel messages can be followed by using the program dmesg. The problem is that it doesn’t do automatic refreshing. You can accomplish this with ‘watch -n 3 “dmesg | tail -n 15” ’, where the 3 is the number of seconds between refreshes, and the 15 is the number of lines to display.

Step 12

Network statistics
The next area you will want to monitor is networking. One utility you can use is netstat. To see all of the current connections on your server, you can use ‘netstat -at | grep -v LISTEN’. This is non-refreshing, so again you will likely want to pass it to watch in order to get an updating output.


       With tmux, you can create a monitoring system allowing you to check on your server remotely and get the perfect overview of what’s happening. Joey Bernard explains how…


Step 13

Disconnecting tmux
The next powerful feature of tmux is the ability to take your session and detach it from the console that you are currently using. To do this, you can use the shortcut key ‘C-b d’. This puts tmux into the background, allowing you to logout of the server if you wish. The great thing is this also works if your connection simply dies, too.

Step 14

Reconnecting tmux
Now that you have a tmux session set up that is monitoring all of the parts of your server that you are interested in, you may want to check in on it. You can log into your server and simply reattach to the existing tmux session with ‘tmux attach-session’.

Step 15

Byobu
There is an alternative program available called Byobu. This program is actually a wrapper around both tmux and screen. It provides a prettier interface to tmux, including a more detailed, two-line status bar at the bottom of the screen. This improved status bar will give you more information, like battery level, CPU frequency and temperature, and even whether there are updates available for your system. These extras are all configurable, and there is even the option of creating a custom notification. You should consider checking Byobu out as a ‘tmux+’ option for your monitoring setup.

Step 16

Naming windows
Once you have your monitoring windows set up, you will likely want to name them so that they are easier to manage. You can do so with the ‘C-b ,’ shortcut. This will rename the current window, and this new name will appear in the list at the bottom of the screen.

Step 17

Configuration files
All of the commands you have used so far to create your monitoring session manually can be done automatically through the use of a configuration file. Each of the shortcuts has an equivalent long command which can be used in the configuration file.

Step 18

Creating windows
To create a new window, you need to add the line ‘new-window’ to the configuration file. When you create this new window, you can give it a target of a current window whose index is where your new window will be inserted.

Step 19

Starting top
Another important option to the ‘new- window’ command is a shell command to execute upon launching the new window. This is where you would place the command to start up ‘top’ within your new window.

Step 20

Naming windows
Naming windows is done through the option ‘-n NAME’. This is important as it makes managing the windows easier. This name gets used to label the window, and it also gets used when you target a window with some particular command through the ‘-t TARGET’ option.

Step 21

Creating panes
To create a pane, you will need to know which window you want to do so in. You can use the ‘split-window’ command, with either the ‘-h’ option for horizontal splitting or ‘-v’ for vertical splitting. Panes are identified through their 0-based index in the current window.

Step 22

Starting tail in a pane
To start up a program in your new pane, you can add the command to the end of your ‘split-window’ tmux command. You can change this at runtime with the tmux command ‘respawn-pane -k -t TARGET-PANE command’, which will kill the current process and start up your new one.

Step 23

Loading a configuration file
After all of this work, you should have a configuration file that will load your entire monitoring session. To do so, you can save it to the default filename ‘~/.tmux.conf’, or you can save it to another filename and load it with ‘tmux -f filename’.

Step 24

What else can you do?
This has only been a start. You can take this and add your own monitoring programs to your tmux session to help in your system administrator duties. You can now connect to your system on a whim and see what is going on in a matter of moments.