Christiaan008 Wiki - New pages [en]

Plotting schools Port Elizabeth using shell scripting

2016-09-27T09:21:16Z

Christiaan008: Created page with "<pre> #Script written on 3 Feb 2016 #Purpose: retrieving coordinates for schools Port Elizabeth #While reading file 'schools1.csv' per line, split by ; character into variabl..."

<pre>
#Script written on 3 Feb 2016
#Purpose: retrieving coordinates for schools Port Elizabeth

#While reading file 'schools1.csv' per line, split by ; character into variable name,tel,phase,suburb,special
while IFS=";" read -r name tel phase suburb special
do

#Take the name of a school, replace white spaces with a + sign and 'To Be Updated and 'Port Elizabeth' with nothing (remove)
#Afterwards place after school name +Port+Elizabeth
poi=$(echo $name | sed "s| |+|g" | sed "s|To Be Updated||g" | sed "s|Port Elizabeth||g" | sed "s|$|+Port+Elizabeth|")

#Do a webrequest to the Google Geocoding API
request=$(curl --silent "https://maps.googleapis.com/maps/api/place/textsearch/xml?query="$poi"&key=insertyourownkey")

#Take the output of the request and only use the information between <formatted_address> and </formatted_address>
formatted=$(echo $request | grep -o -P '(?<=<formatted_address>).*(?=</formatted_address>)')

#Take the output of the request and only use the information between <lat> and </lat>
lat=$(echo $request | grep -o -P '(?<=<lat>).*(?=</lat>)')

#Take the output of the request and only use the information between <lng> and </lng>
long=$(echo $request | grep -o -P '(?<=<lng>).*(?=</lng>)')

#Ouput name, address and coordinates to the screen
echo "School: $name"
echo "Address: $formatted"
echo "Coordinates: $lat , $long"

#Go to sleep for 5 seconds (do nothing for 5 seconds)
sleep 5

#Write the school name, Address and coordinates to the file 'schools_w_coordinates.csv'
echo $name";"$formatted";"$lat";"$long >> schools_w_coordinates.csv

#End the while loop
done < "schools1.csv"
</pre>

Retrieving Flickr photo views using shell scripting

2016-07-12T19:49:21Z

Christiaan008: Created page with "<pre> #!/bin/bash #The purpose of this script is to retrieve the amount of views for your Flickr photos. #Your Flickr API key KEY="" #Your Flickr user ID user="" #Format fo..."

<pre>
#!/bin/bash
#The purpose of this script is to retrieve the amount of views for your Flickr photos.

#Your Flickr API key
KEY=""

#Your Flickr user ID
user=""

#Format for today's date
today=$(date +"%m-%d-%Y")
totalviews="0"

#Format for date in filename
filedate=$(date +"%y%m%d_flickr_stats.txt")
filedateyesterday=$(date --date="1 day ago" +"%y%m%d_flickr_stats.txt")

#If you run as an automated cron job this helps to store files
path="/home/username"

#Get all the IDs of the Flickr photos for certain user using the Flickr API
#Only grep the ID numbers
ids=$(curl -s "https://api.flickr.com/services/rest/?method=flickr.people.getPhotos&api_key="$KEY"&user_id="$user"&per_page=500&format=rest" | grep -o -P '(?<=photo id=").*(?=" owner=)')

#Read the output from variable ids in idsarray, remove all the whitespaces in the ids variable
read -r -a idsarray <<< $ids

#Iterate through the array and for every entry do the following
for id in "${idsarray[@]}"
do
#Request the information of the photo using the photo's ID
info=$(curl -s "https://api.flickr.com/services/rest/?method=flickr.photos.getInfo&api_key="$KEY"&photo_id="$id"&format=rest")

#From the ouput of the API request only take the values for title and views
title=$(echo $info | grep -o -P '(?<=<title>).*(?=</title>)')
views=$(echo $info | grep -o -P '(?<=views=").*(?=" media)')

echo "Photo $title has $views views"

#Get the views for this photo ID from yesterday
photoviewsyesterday=$(cat $path$filedateyesterday | grep "$id" | cut -d '|' -f 4)

#Calculate the difference between todays and yesterdays views for this photo ID
photoviewsdifference=`expr $views - $photoviewsyesterday`

#Write the output to the file
echo "$today|$id|$title|$views|$photoviewsdifference">> $path$filedate

#Add the current views to the total views
totalviews=`expr $totalviews + $views`
done

#Get the total views from yesterday
totalviewsyesterday=$(cat "$path"flickr_total_views.txt | grep "$(date --date="1 day ago" +"%m-%d-%Y")" | cut -d '|' -f 2)

#Calculate the difference betweeen todays and yesterdays total views
difference=`expr $totalviews - $totalviewsyesterday`

#Write the output to the file
echo "$today|$totalviews|$difference" >> "$path"flickr_total_views.txt
</pre>

Dynamic splitting output into files while processing

2015-11-08T21:35:08Z

Christiaan008: Christiaan008 moved page Dynamic splitting output into files while processing to Dynamically splitting output into files while processing

<pre>#!/bin/bash

#The purpose of this script is to dynamically split the output of generated entries into different files
#Set the variable values before the while loop so that there is some input
start="0"
amount="5" #The amount to use to split
begin=$(echo $start) #The first time this will be the start value
end=$(expr $amount - 1) #The first time this will be the amount minus 1 value

#This loop will continue until the variable $start will hit 10
while [[ $start -lt 20 ]]
do
#Check if the expected file exists (only necessary for the first run)
if [[ -f "numbers"$begin"_"$end".txt" ]];
then
#Count the amount of lines in the current file
lines=$(wc -l < "numbers"$begin"_"$end".txt")
else
lines="0"
fi

#If the result of the amount of lines in the current file is less then the number that is set add the entry to the current file
if [[ $lines -lt $amount ]]
then
echo $start >> "numbers"$begin"_"$end".txt"
#Else the amount of lines in the current file is more then the amount that is set in the value $amount
else
#Update the variables $begin and $end by adding the value of variable $amount
begin=$(expr $begin + $amount)
end=$(expr $end + $amount)
#Add current entry to new file
echo $start >> "numbers"$begin"_"$end".txt"
fi

#Update the variable $start by 1 otherwise the loop will run indefinitely
start=$(expr $start + 1)
done</pre>

Scripting

2015-11-06T17:38:14Z

Christiaan008:

[[Dynamically splitting output into files while processing]] 
[[Retrieving Flickr photo views using shell scripting]] 
[[Plotting schools Port Elizabeth using shell scripting]]

Other

2014-09-13T18:57:03Z

Christiaan008:

== Useful links ==
YouTube 
* [http://stackoverflow.com/questions/2068344/how-do-i-get-a-youtube-video-thumbnail-from-the-youtube-api How do I get a YouTube video thumbnail from the YouTube API? - stackoverflow.com] 

== Inspiring talks and other interesting videos ==
* [http://www.ted.com/talks/hugh_herr_the_new_bionics_that_let_us_run_climb_and_dance The new bionics that let us run, climb and dance - ted.com] 
* [https://www.youtube.com/watch?v=UF8uR6Z6KLc Steve Jobs' 2005 Stanford Commencement Address - youtube.com] 
* [http://www.ted.com/talks/simon_sinek_how_great_leaders_inspire_action How great leaders inspire action - ted.com] 
* [https://www.youtube.com/watch?v=wvhW8cp15tk Steve Jobs and Bill Gates Together at D5 Conference 2007 - youtube.com]

Configurations

2014-09-06T11:33:07Z

Christiaan008:

Configuring Debian 
* [https://www.digitalocean.com/community/tutorials/how-to-add-delete-and-grant-sudo-privileges-to-users-on-a-debian-vps How To Add, Delete, and Grant Sudo Privileges to Users on a Debian VPS - digitalocean.com] 
* [https://wiki.debian.org/iptables iptables Debian - debian.org] 
* [http://www.cyberciti.biz/tips/linux-iptables-examples.html Linux: 20 Iptables Examples For New SysAdmins - cyberciti.biz] 
* [http://www.iptables.info/en/iptables-gui.html Overview of GUI's available for building iptables rules - iptables.info] 
* [http://wiki.centos.org/HowTos/Network/IPTables iptables CentOS - centos.org] 
* [https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-using-iptables-on-ubuntu-14-04 How To Set Up a Firewall Using IPTables on Ubuntu 14.04 - digitalocean.org] 

Configuring Apache for multiple virtual hosts 
* [https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-12-04-lts How To Set Up Apache Virtual Hosts on Ubuntu 12.04 LTS - digitalocean.com] 

Configuring WordPress for use of multiple domains 
* [http://codex.wordpress.org/Create_A_Network Create A Network - wordpress.org] 
* [https://www.digitalocean.com/community/tutorials/how-to-set-up-multiple-wordpress-sites-using-multisite How To Set Up Multiple WordPress Sites Using Multisite - digitalocean.com] 

Configuring SSL 
* [https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-with-a-free-signed-ssl-certificate-on-a-vps How To Set Up Apache with a Free Signed SSL Certificate on a VPS - digitalocean.com] 
* [https://www.digicert.com/ssl-support/apache-multiple-ssl-certificates-using-sni.htm Using Multiple SSL Certificates in Apache with One IP Address - digicert.com] 

Configuring redirect on webserver 
* [http://stackoverflow.com/questions/6880903/redirect-ip-address-to-domain-name Redirect IP address to domain name - stackoverflow] 
* [http://kb.mediatemple.net/questions/85/Using+.htaccess+rewrite+rules Using .htaccess rewrite rules - mediatemple.net] 

Configuring Mediawiki 
* [http://t-machine.org/index.php/2009/06/26/making-mediawiki-secure-and-fixing-some-config-annoyances/ Making MediaWiki secure (and fixing some config annoyances) - t-machine.org] 
* [http://www.mediawiki.org/wiki/Extension:Email_notification Extension:Email notification - mediawiki.org] 

Configuring WordPress 
* [https://rtcamp.com/tutorials/php/test-email-sending/ Checking if PHP/WordPress can send mails - rtcamp.com] 
* [http://codex.wordpress.org/Brute_Force_Attacks Brute Force Attacks - wordpress.org] 
* [http://codex.wordpress.org/Hardening_WordPress Hardening WordPress - wordpress.org] 
* [http://colorlabsproject.com/tutorials/remove-slugs-custom-post-type-url/ Remove The Slugs from Custom Post Type URL - colorlabsproject.com] 

WordPress plugins 
* [https://wordpress.org/plugins/all-in-one-wp-security-and-firewall/ All In One WP Security & Firewall - wordpress.org] 
* [https://wordpress.org/plugins/enhanced-media-library/ Enhanced Media Library - wordpress.org] 
* [https://wordpress.org/plugins/better-wp-security/ iThemes Security (formerly Better WP Security) - wordpress.org] 
* [https://wordpress.org/plugins/social-share-button/ Social Share Button - wordpress.org] 
* [https://wordpress.org/plugins/wordpress-https/ WordPress HTTPS (SSL) - wordpress.org]

Installations

2014-09-06T11:32:37Z

Christiaan008:

== Useful links ==
Installation of Apache, MySQL, PHP 
* [https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-debian How To Install Linux, Apache, MySQL, PHP (LAMP) stack on Debian - digitalocean.com] 

Installing WordPress 
* [https://www.digitalocean.com/community/tutorials/how-to-install-wordpress-on-debian-7 How To Install WordPress on Debian 7 - digitalocean.com] 

Installing Mediawiki 
* [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu Running MediaWiki on Ubuntu - mediawiki.org] 

Installation of the e-mailserver (Postfix, Dovecot) 
* [https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-e-mail-server-with-dovecot How To Set Up a Postfix E-Mail Server with Dovecot - digitalocean.com] 
* [http://serverfault.com/questions/42519/how-to-correct-postfix-relay-access-denied How to correct Postfix' 'Relay Access Denied'? ]

Main Page

2014-09-05T13:09:54Z

Christiaan008:

Welcome to the Christiaan008 Wiki page.

[[Installations]] 
[[Configurations]] 
[[Other]] 
[[Scripting]]