getNowAsTimestampString

public String getNowAsTimestampString() {
Calendar nowCalendar = Calendar.getInstance();
java.util.Date myDate = nowCalendar.getTime(); // ex: Thu Aug 09 13:20:36 EDT 2007
SimpleDateFormat sdf = new SimpleDateFormat("yyyy.MM.dd_HH.mm.ss");
return sdf.format(myDate);
}

Setting up a gateway - fresh OpenSUSE set up

ping google.com
connect: Network is unreachable

Grrr...


cat /etc/resolv.conf
echo "alias net-pf-10 off" >> /etc/modprobe.conf
echo "alias ipv6 off" >> /etc/modprobe.conf
ip route show all
route add default gw
ip route show all
ping google.com

Fixed!

How to back up web accessible files using wget, and to remove files older than 30 days

1. Script to back up files over wget:

#!/bin/bash
#
# This script grabs all the my-server.alanlupsha.com/logs/ files and
# backs them up on the local server in /home/
#
# @author Alan Lupsha 12/19/2011
#

# keep track of the start time
STARTTIME="Started backup at: `date`"


#########################################################################
# You should change these entries
#
# make sure to create the directory first
# ex: "mkdir /home/lupsha/backups"
BACKUPDIR=/home/lupsha/backups

# save text to a temporary file
EMAILFILE=/tmp/email.txt

# update this
EMAILADDRESS=my-email-address@my-domain.com
#########################################################################

# clean up email file
rm "$EMAILFILE"

# save file in this directory
cd "$BACKUPDIR"

LOC="`pwd`"
if [ "$LOC" != "$BACKUPDIR" ]; then
echo "ERROR, I should be in $BACKUPDIR, but I'm not! Exiting...";
exit 1
fi

# download files
wget -e robots=off --recursive --no-directories \
--no-host-directories --level=1 --no-clobber \
http://my-server.alanlupsha.com/logs/


# save timestamp
echo "$STARTTIME" >> $EMAILFILE
echo "\r\n" >> $EMAILFILE
echo "Ending pass logs backup at `date`" >> $EMAILFILE

# send an email with the result
cat $EMAILFILE | mail -s "Ran pass logs backup" $EMAILADDRESS

2. Save the script and run it from a cron job, only week days, at 7:30 AM

sudo crontab -e

# run script every week day at 7:30AM
30 07 * * 1-5 /home/lupsha/backup-logs-script.sh

3. On the server where the files are stored, remove the files that are older than 30 days:

cd /var/www/html/myfiles
find . -type f -mtime +30 -exec rm -- {} \;

How to convert video to a lossless format using ffmpeg!

How to convert a .3gp video to .flv:
ffmpeg -i a.3gp -ar 22050 -ab 32k -f flv a.flv


How to convert to almost lossless format:
ffmpeg -i a.3gp -an -f yuv4mpegpipe b.avi

(Careful, an 11MB 3gp file becomes a 1.6GB avi file!)

Renaming directories and files

Supposed you have the following directory structure:

/home/user/files/A/
/home/user/files/A/1/
/home/user/files/A/2/
/home/user/files/A/3/
/home/user/files/B/
/home/user/files/B/37/
/home/user/files/B/38/
/home/user/files/B/39/

and you wish to concatenate the subdirectories to the parent directories (ex: "A - 1", "A - 2", "B - 38"), to result in the following structure:

/home/user/files/A - 1/
/home/user/files/A - 2/
/home/user/files/A - 3/
/home/user/files/B - 37/
/home/user/files/B - 38/
/home/user/files/B - 39/


First,
cd /home/user/files

Next, allow spaces in file names:
IFS=$(echo -en "\n\b")

Then:
for dir in `ls`; do cd "$dir"; for files in `ls`; do mv "$files" "$dir - $files"; mv "$dir - $files" .. ; done; cd .. ; echo "rm -Rf $dir" >> /tmp/cleanup.sh ; done

Take a look at the generated cleanup file:
cat /tmp/cleanup.sh

If it looks ok, make it executable:
chmod u+x /tmp/cleanup.sh

and execute it:
/tmp/cleanup.sh

How to convert .m2ts videos to .avi, using a script

The problem:
I have a bunch of vacation videos located in: /mnt/drive2/vacation_videos/ , but they are all .m2ts format, and I want them to be in .avi format

The solution:
- Create a generic script "convert.sh" which converts a .m2ts file to .avi
- Create a script "vacationVideos.sh" with all the custom vacation videos, and call the convert.sh script
- Run this as a background process, which allows you to log out of the session without interrupting the process. i.e. "nohup shellscriptname &"


1. Generic conversion script. Save as: nano /usr/sbin/convert.sh

---

#!/bin/bash
# AlanLupsha: this script converts .m2ts files to .avi files
# If you do not have mencoder, run sudo apt-get install mencoder
if [ -z $1 ] ; then
        echo "syntax: convert.sh file.m2ts"
        RETURNCODE=1
        exit $RETURNCODE
fi
mencoder "$1" -ofps 23.976 -ovc lavc -oac copy -o  "$1.avi"

---

2. custom script with the names of all my vacation videos. Save as: nano ~/vacationVideos.sh

---

#!/bin/bash
# This "vacationVideos.sh" script copies my vacation videos into a
# temporary directory and then invokes the convert.sh
# script which converts .m2ts format to the .avi format

MAINPATH=/mnt/drive2/vacation_videos/
TMP=/dev/shm
cd "$TMP"

FILE="2011.grandparents.m2ts"
cp "$MAINPATH/$FILE" "$TMP"
# convert the file from .m2ts to .avi (takes a long time)
convert.sh "$FILE"
# copy the converted .avi file back to the drive
mv "$TMP/$FILE.avi" "$MAINPATH"
# remove the temporary copy of the .m2ts file
rm "$TMP/$FILE"


FILE="2010.SouthFlorida.m2ts"
cp "$MAINPATH/$FILE" "$TMP"
# convert the file from .m2ts to .avi (takes a long time)
convert.sh "$FILE"
# copy the converted .avi file back to the drive
mv "$TMP/$FILE.avi" "$MAINPATH"
# remove the temporary copy of the .m2ts file
rm "$TMP/$FILE"


FILE="2006.DriveAcrossTheUSA.m2ts"
cp "$MAINPATH/$FILE" "$TMP"
# convert the file from .m2ts to .avi (takes a long time)
convert.sh "$FILE"
# copy the converted .avi file back to the drive
mv "$TMP/$FILE.avi" "$MAINPATH"
# remove the temporary copy of the .m2ts file
rm "$TMP/$FILE"

echo "Done converting!"

---

3. How to run:

---

ssh myname@linuxserver
nohup ~/vacationVideos.sh &

To see the progress:
tail -f nohup.out

To see the process:
ps aux | grep mencoder

To kill the process:
killall mencoder

Enjoy.

How to set up Subversion in Ubuntu

sudo adduser subversion

sudo mkdir /home/svn
sudo svnadmin create /home/svn/dev

# change ownership
sudo chown -R subversion:www-data /home/svn/dev

# make the dir group write-able
chmod -R g+w /home/svn/dev/

# add yourself to the www-data group
sudo usermod -a -G www-data alan



sudo apt-get install apache2
sudo apt-get install libapache2-svn

sudo nano /etc/apache2/mods-available/dav_svn.conf

add at the end:


DAV svn
SVNParentPath /home/svn
SVNListParentPath On
AuthType Basic
AuthName "Subversion Repository"
# AuthUserFile /etc/subversion/passwd

Require valid-user




Save.

sudo /etc/init.d/apache2 restart


# Add your users in subversion, create their accounts:
sudo nano /home/svn/dev/conf/passwd

alan=123456
mary=654321


# For subversion over http, also add the same users to the
# subversion passwd file, first time use -c to "create" it.
sudo htpasswd -c /etc/subversion/passwd alan
sudo htpasswd /etc/subversion/passwd mary


# Empty out config file
sudo su -
echo "" > /home/svn/dev/conf/svnserve.conf

# Edit file and add next entries:
sudo -u subversion nano /home/svn/dev/conf/svnserve.conf


[general]
anon-access = none
auth-access = write
password-db = passwd
realm = digitalagora
[sasl]

# Save, exit.

# Install xinetd, and let it take care of running the subversion server
sudo apt-get install xinetd

# edit a new svn file
sudo nano /etc/xinetd.d/svn

# Paste these contents:

service svn
{
port = 3690
protocol = tcp
socket_type = stream
wait = no
disable = no
user = subversion
server = /usr/bin/svnserve
server_args = -i -r /home/svn/dev

#ALAN - increase incoming connections per second
# X connections per second, 1 sec to reset after overload
cps = 3000 1

# if in /var/log/messages you see: FAIL: svn per_source_limit from=IP...
per_source = UNLIMITED
}

# Save the file, exit.

# start the xinetd service
sudo service xinetd restart

# check that subversion is running, you should not get an error from this command (use your

username/pass)
svn --non-interactive --username alan --password abc12345 list svn://localhost/



# ===================================================
# ALTERNATIVELY, IF YOU DO NOT WANT TO USE XINETD:
# Start the server manually...
svnserve -d -r /home/svn/
#
# Check that it's running
ps aux | grep svn
#
You should see something like: svnserve -d -r /home/svn/
# to stop svnserve, run
ps aux | grep svn
killall svnserve
ps aux | grep svn
# ===================================================



Test the installation

# create project
svn --non-interactive --username alan --password abc12345 -m "new" mkdir

svn://localhost/dev/testproject

# check out project
cd ~
svn --non-interactive --username alan --password myPassw0rd checkout

svn://localhost/dev/testproject

# create new files in checked out project
touch ~/testproject/newfile1.txt
touch ~/testproject/newfile2.txt
touch ~/testproject/newfile3.txt

# go into project
cd ~/testproject
svn add *

# commit the files
svn commit -m "new" *

Adding newfile1.txt
Adding newfile2.txt
Adding newfile3.txt
Transmitting file data ...
Committed revision 2.




=================================
a2ensite default-ssl
a2enmod ssl
sudo /etc/init.d/apache2 restart
sudo make-ssl-cert generate-default-snakeoil --force-overwrite
=================================

Check that it's working:
http://localhost/

Ubuntu - simple setup after install

0. change root password
su -
passwd root

1. static network
Check the network interface:
ifconfig

Change to static network:

auto eth0
iface eth0 inet static
address 10.1.10.99
netmask 255.255.255.0
network 10.1.10.0
broadcast 10.1.10.255
gateway 10.1.10.1

sudo /etc/init.d/networking restart


2. Install SSH server:
sudo apt-get install openssh-server


3. Disable root logins:
nano /etc/ssh/sshd_config

Change to say "no":
PermitRootLogin no

Save file, exit.
Start up ssh:
/etc/init.d/ssh restart

4. test by logging in from another server:
ssh user@newserver

5. Set up secondary drives using UUID:

sudo mkdir /mnt/d1
sudo mkdir /mnt/d2
sudo mkdir /mnt/d3
sudo mkdir /mnt/d4

Next, we need to list all the drives and mark down the UUID of each drive, and make sure you know which drive is sda1, sdb1, sdc1, and so on.

# list all drives
sudo fdisk -l

# next, list them by uuid
ls -l /dev/disk/by-uuid/

Edit fstab and start adding entries. For each drive, copy and paste its UUID and associate it with a mount point, ex: /mnt/d1, etc.

sudo nano /etc/fstab

# sda1
UUID=ba20caf8-6ef0-42e7-9e45-4187bfa8e543 /mnt/d1 auto defaults 0 2
# sdb1
UUID=315310c3-b3f2-4092-a7fa-eb944084e335 /mnt/d2 auto defaults 0 2
# sdc1
UUID=e041f16e-a009-4390-97f9-2daf29c0c505 /mnt/d3 auto defaults 0 2
# sdd1
UUID=2a236394-5010-4742-9ef8-1a5b2d1b74fa /mnt/d4 auto defaults 0 2

Save fstab file, close it.

Mount all file systems in /etc/fstab, run:
sudo mount -a
(fix any errors, try again, until it works)

List all drive contents,
ls /mnt/d1
ls /mnt/d2
ls /mnt/d3
ls /mnt/d4

I personally create directories on each drive so that I can identify the drives later. For example, on the root of each drive, I create a directory such as: "this_is_drive_1"

6. set up Samba

#Install samba - for network file sharing
sudo apt-get install samba

#create group
sudo groupadd samba

#create a system user who should have samba access
sudo adduser sambauser

#add this user to the samba group
sudo usermod -a -G samba sambauser

# add the user who can access samba shares
sudo smbpasswd -a sambauser
(enter their password)

#edit conf file
sudo nano /etc/samba/smb.conf

# my drives
[d1]
comment = This is the /d1 shared drive
path = /mnt/d1
browseable = yes
read only = no
guest ok = no
writable = yes
admin users = sambauser
write list = sambauser
create mask = 0775
directory mask = 0775
public = yes

[d3]
comment = This is the /d3 shared drive
path = /mnt/d3
browseable = yes
read only = no
guest ok = no
writable = yes
admin users = sambauser
write list = sambauser
create mask = 0775
directory mask = 0775
public = yes


Save the file.

#restart samba
sudo service smbd restart

# list the shares from another PC (from Windows, start, run \\IP_of_server)
smbclient --username=sambauser -L 10.1.10.99
(you should see the drives being listed)

You just bought a new 1 Terrabyte drive for your Ubuntu server. Now what?

After installing the RAID card for the SATA drive, and after seeing the drive at boot-up time, you're ready to partition the drive, format it, and mount it by id, and then use it.

1. List drives:
sudo fdisk -l

2. If drive is not formatted but it is listed, format it (ex: drive is /dev/sdb):
sudo fdisk /dev/sdb
p (print partition information)
n (create new partition)
p (primary partition)
1 (partition #1)
[enter] (accept first cylinder default, which is 1)
[enter] (accept last cylinder default, whatever that is)
w (write partition information, and then the program will exit)

Verify that a partition has been created (it's not yet formatted), do a list:
sudo fdisk -l

You should see the new partition listed, and a number added to the device name. For example, the drive is /dev/sdb and the partition became /dev/sdb1

3. Format the partition (Be very careful that it's the correct partition, in this example /dev/sdb1):

mkfs.ext3 /dev/sdb1

Wait for the formatting to be over.

4. List drives by UUID:
ls -l /dev/disk/by-uuid/
Note down the UUID of the disk that you've just formatted, for example sdb1.
The UUID is a long sequence of alphanumeric characters.

5. Create directory where to mount drive (disk 1 = /mnt/d1)
sudo mkdir /mnt/d1

6. Copy UUID of drive in fstab so that it gets mounted every time the server is booted:
sudo nano /etc/fstab

# My 1TB drive on /dev/sdb1
UUID=91f86f8a-4b59-4b67-b62b-0f2a3c2b235c /mnt/d1 auto defaults 0 2

6b. Optionally, if you just want to mount the drive now without adding it to the automatic mounting in the /etc/fstab file, then you can mount it manually, also by UUID:
sudo mount -U 91f86f8a-4b59-4b67-b62b-0f2a3c2b235c
or
sudo mount /dev/sdb1 /mnt/d1


7. Restart server, if you have to. (you shouldn't have to)
sudo shutdown -r now

Now the drive is available in /mnt/d1. Check the space: df -h /mnt/d1

How to fix annoying Ubuntu Nautilus errors such as... "Nautilus cannot handle burn locations"

"Nautilus cannot handle burn locations"
"Nautilus cannot handle COMPUTER actions... and such...


sudo apt-get remove gvfs
sudo apt-get install gvfs

Restart the system.
The whole Nautilus look and feel is different, and everything starts working, even samba shares: smb://me:passwd@192.168.0.1/myshare, etc...

Done.