Automatically Download All The Wallpapers From National Geographic [MAC Version]
In my travels today i found a great post from http://www.webupd8.org Its a set up scripts for linux that will download all the Wallpapers from Nation Geopraphic.
Well, it was Linux only and didn't work nativity on my Mac, so i hacked it up and made it Mac Friendly
Original Post: http://www.webupd8.org/2009/11/automatically-download-all-wallpapers.html
My Scripts:
2008 Wallpapers : http://bjcpgd.cias.rit.edu/natgeo/natgeo2008.sh
2009 Wallpapers : http://bjcpgd.cias.rit.edu/natgeo/natgeo2009.sh
To run these scripts. download them to a folder of your choice, remember all the wallpapers will download to that directory.
Open up the Terminal.app from Applications > Utilities
Work your way into your directory structure and you will have to:
chmod +x /natgeo2008.sh
and
chmod +x /natgeo2009.sh
When you are ready just
./natgeo2008.sh
All credit for this script goes back to original Author, i'm just the guy who ported it to Mac.
modrewrite to combat spam attacks
This is a simple ModRewrite i use to combat the file upload, then expand attack that is fairly common with poorly created upload scripts
Basically this reads, if the request or the refferer contains any of the bad words, redirect it to a script called 'nospam.php' (in the root of your server) and give a 404 error, so the page will be removed from search engines.
RewriteCond %{THE_REQUEST} tnb|torrent|bad-credit|4u|adipex|advicer|baccarrat|blackjack|bllogspot|booker|byob|car-rental-e-site|car-rentals-e-site|carisoprodol|casino|casinos|chatroom|cialis|coolcoolhu|coolhu|credit-card-debt|credit-report-4u|cwas|cyclen|cyclobenzaprine|dating-e-site|day-trading|debt-consolidation|debt-consolidation-consultant|discreetordering|duty-free|dutyfree|equityloans|fioricet|flowers-leading-site|freenet-shopping|freenet|gambling-|hair-loss|health-insurancedeals-4u|homeequityloans|homefinance|holdem|holdempoker|holdemsoftware|holdemtexasturbowilson|hotel-dealse-site|hotele-site|hotelse-site|incest|insurance-quotesdeals-4u|insurancedeals-4u|jrcreations|levitra|macinstruct|mortgage-4-u|mortgagequotes|online-gambling|onlinegambling-4u|ottawavalleyag|ownsthis|palm-texas-holdem-game|paxil|penis|pharmacy|phentermine|poker-chip|poze|pussy|rental-car-e-site|ringtones|roulette|shemale|shoes|slot-machine|texas-holdem|thorcarlson|top-site|top-e-site|tramadol|trim-spa|ultram|valeofglamorganconservatives|viagra|vioxx|xanax|zolus
RewriteRule (.*) nospam.php [R=404,L]
RewriteCond %{HTTP_REFERER} tnb|torrent|bad-credit|4u|adipex|advicer|baccarrat|blackjack|bllogspot|booker|byob|car-rental-e-site|car-rentals-e-site|carisoprodol|casino|casinos|chatroom|cialis|coolcoolhu|coolhu|credit-card-debt|credit-report-4u|cwas|cyclen|cyclobenzaprine|dating-e-site|day-trading|debt-consolidation|debt-consolidation-consultant|discreetordering|duty-free|dutyfree|equityloans|fioricet|flowers-leading-site|freenet-shopping|freenet|gambling-|hair-loss|health-insurancedeals-4u|homeequityloans|homefinance|holdem|holdempoker|holdemsoftware|holdemtexasturbowilson|hotel-dealse-site|hotele-site|hotelse-site|incest|insurance-quotesdeals-4u|insurancedeals-4u|jrcreations|levitra|macinstruct|mortgage-4-u|mortgagequotes|online-gambling|onlinegambling-4u|ottawavalleyag|ownsthis|palm-texas-holdem-game|paxil|penis|pharmacy|phentermine|poker-chip|poze|pussy|rental-car-e-site|ringtones|roulette|shemale|shoes|slot-machine|texas-holdem|thorcarlson|top-site|top-e-site|tramadol|trim-spa|ultram|valeofglamorganconservatives|viagra|vioxx|xanax|zolus
RewriteRule (.*) nospam.php [R=404,L]
This simple addition is a nice little tool in your web server admin toolbelt.
SPAM SPAM GO AWAY!
It seems the new trend for successfully exploited weak web applications is that they will only show their seedy content to google or other search engines.
They will only be triggered to show their content when they are accessed by GoogleBot from a Google IP (yeah they are getting that specific).
When you click and view a page that google say's is full of Viagra spam, you won't see anything, its tricky and VERY frustrating and hard to troubleshoot.
So far, the common sign i've seen of successful exploits have been:
1. .bak files (installed as wordpress plugins, you have to scour your 'active_plugins' field in the database
2. .pngg .giff .jpgg and .old files, trying to upload malicious PHP and get around unsecure uploaders
3. the use of the base64_decode PHP function, while there are legit uses for this function, it can be a sign of a baddie
4. Use of the 'eval' function in PHP. Also, legit uses are out there, but i've seen it used for the dark side of the force.
5. a 'WordPress' user in your WordPress user table.
If you want to scan a *nix system for the file names i've found to be 'bad' use the following commands.
find -name *_old.php*
find -name *.php.jpgg
find -name *.php.giff
find -name *.php.pngg
To look for those functions i talked about your can use your friend 'grep'
grep -inrH "eval(base64_decode(" <your dir here>
grep -inrH "gzinflate(base64_decode(" <your dir here>
For anyone interested I've recently installed mod_security with their core rule sets on our Apache webserver and after tweaking the config files and creating some white-lists I have be able to ward off a number of baddies and exploit attempts.
http://www.modsecurity.org/
Its worth the hassle of setting it up. It also has a 'detection only' mode which does a great job letting you know what you have running and tweak the rules before it starts to block requests.
Well… that was an interesting night!
Tonight was supposed to be a very simple hard drive copy to a server, virtualize a physical server, get a beer and go home... I figured, i'd be at home by 6 or 7pm... well... Technology has a way of really screwing with you. So here's a little story about my stupid night.
So 5:00pm goes around and i turn off the old desktop class machine that has been used as a server for the better part of 5 years. The plan is to take the hard drive will all the unamed organizations data off of it, put it on to our super fast secure awesome server, and then call it a night, if I had time i was going to get rid of the old physical hardware once and for all and put the server on our virtual server cluster, no brainer, i've done this kind of thing a million times before.
Here is where the first snag came, i got the hard drive, put it in our hard drive to firewire adaptor, and i couldn't get it to mount. Hell, i couldn't get it to spin... finally after some fiddling and adding the external power plug for the adaptor i got it going, i started the file copy, i thought i was done. I promptly went to play COD4 and blow some 12 year noobs up. I turned around to check on the status of the file copy, and i noticed the drive wasn't spinning. "Strange" I think to myself. I try remounting the drive over and over again, and it just won't go, this is already pissing me off. I put the drive on my Windows XP machine and it mounts after a long time, and i start the file copy again, and the drive stops spinning AGAIN. So by now i'm pissed. Its fairly obvious the drive isn't working properly, and this explains the slow performance the organization had been seeing on their old file server. So what do I do? Only what any sane IT professional would do with a semi dead hard drive, i put it in the freezer. Yes you read that right, i put it in the freezer. Its a religous thing, the hard drive in freezer trick. Some people say its hog-wash, other swear by it. In our office, its saved our butts a few time, so "I want to believe"
At this point Jay and I decide that while the hard drive is on ice, we will take a walk down to the parking lot and pick up our vehicles from the far corner of the lot and bring them closer, if anything to get outside and get fresh air and not be around stupid dead hard drives... and then we saw something that was just... strange... I think the picture will speak for its self. A box truck, inbetween the two levels of parking.
So after wondering WTF was going on the with the box truck we headed back in to check on our frozen hard drive. We hooked it up, and BAM it started working right off the bat, real fast and everything... and then about 5 minutes into the copy it failed again. At this point i was starting to loose faith that we would get this going. So, we froze it again. Tried it again, and it would go for a while, then fail. This is the point where desperation is starting to set in. I really want to get this server finished by start of business Tuesday. So i thought to myself, "What would happen if i ran the hard drive from INSIDE the freezer"
Yes, thats a power cable, and firewire cable going into the office fridgem which we've moved next to my desk. The sick part? IT WORKS. The hard drive is running, and actually copying data. Its still a bit slow, but the hard drive is still working. I just don't understand, but whatever. ITS WORKING!!
Yup, so that was my night, a hard drive, in a static bag, in the freezer.
So, have you ever had any luck with the hard drive in a freezer? Have you ever gone to this lengths? That was one heck of a night at the office.
Now, time for a blast from the past, an mixer board and outboard rack:
Logwatch + OS X
I needed to install logwatch a OS X machine for work, and i figured i would just document the process.
At this point just use all the defaults
Thats is... logwatch is installed as you can see by running
sudo logwatch
(cias staff at this point look to our internal wiki)
Now you just need to tweak your /etc/logwatch/conf/logwatch.conf file to suit your needs
Also if you want it to run everynight at midnight put a script in /etc/periodic/daily
Netboot Across Subnets
Rob wrote a great article on how to Netboot your macs across multiple subnets, like we do here at RIT
Force SSL connection using PHP
When place before all other output on your PHP script this little snipped will force you PHP page to be presented over SSLThis is very useful on things like Login forms.
if($_SERVER['SERVER_PORT'] != '443') {
//Force SSL upon this page
header("Location: https://".$_SERVER['HTTP_HOST'].$_SERVER['PHP_SELF']);
}
Logwatch & RSS – A Perfect Union
Here at CIAS we have a bunch of linux servers, and we run logwatch on them to give us a daily look at our servers.
There are few options with logwatch and how you want to get the information, you can have it email you, which for one or two servers that is pretty standard, but we are managing 10+ servers, and i don't want to clog up my email and i don't want to use my inbox as a way to archive all of our logwatch reports.
So what did I come up with?
On The Server
Everything that happens here is using a simple upload script i created to facilitate the aggregation of the scripts. Machine's logwatch runs as a cron scripts and outputs its report to a directory on the machine. Right after that script runs another scripts run and copies the logwatch report to a central web server.
On the Web Server
The 'logwatch' app is just a nice simple front end to a bunch of directories from this structure
archive
server1
12-3-07
logwatch.txt
12-2-07
logwatch.txt
server2
12-3-07
logwatch.txt
I created a nifty little PHP class that takes care of this structure, and allows easy access to the files through code.
RSS and <insert favorite news reader here>
Cool, so now i have a nifty little web app that organizes our logwatch reports... so whats the next step?
I already run NetNewsWire on my Mac and RSS would be a perfect solution for seeing updated logwatch reports as they come in from the servers. So create the RSS is was pretty easy...
1. use the PHP class
2. output in RSS format the last 5 logwatch reports for the specified server
3. RSS MONEY!!
I'm pretty happy with this elegant solution. I always have a hard time remembering to 'go' to a web app... I always keep NetNewsWire open. So this solution makes me a better System Administrator
Wildcard Certificate on cias.rit.edu
Earlier today we installed a wildcard certificate on cias.rit.edu.
What does this mean?
A: Before today, if you went to https://bjcpgd.cias.rit.edu (over SSL) you would be using the certifcate for cias.rit.edu, and in the eyes of your browser bjcpgd.cias.rit.edu and cias.rit.edu are two totally seperate domains. The installation of this wildcard certifcate now means that the certificate will cover and be valid for all sub-domains of cias.rit.edu. This means you can now easily use SSL to protect your web applications on cias.rit.edu.
Do I have to change anything to use this certificate?
A: Not at all! The certificate is active and valid for all accounts under cias.rit.edu. You may need to tweak your application to make sure its point to https:// in the areas that you want protected, but that is a very application specific issue.
What else has the cias tech team been doing to help secure our applications and data on cias.rit.edu?
A: In the past we have allowed all users to be able to browse other users directories. We've been slightly uncomfortable with this scenario for a while. Yesterday we made a change to everyone's home directory that will keep prying eyes out, yet still allow your web applications to 'roam free'. We specificly did the follow on each users home directory:
chgrp www-data /home/<username> chmod 750 /home/<username>
What exactly did that just do? First off we made the group owner for you home directory the web server's user. So you are the owner of your directory, and www-data is the group. The chmod of 750 means, The owner can read,write and execute. The group can read and write, everyone else has NO ACCESS.
Apache Errors
Tonight i encountered a new error in Apache i've never seen before:
[Thu Feb 28 17:54:32 2008] [notice] child pid 2602 exit signal File size limit exceeded (25)
I saw MANY of these errors, and Apache wouldn't return any pages to the browser.
Here to find out, in 32bit Linux, there is a 2GB file size limit.. when this file size limit is reached on an Apache Log file..... these errors start to crazy.. and breaks Apache.
Lesson learned
So... with that said, once we get all 64bit ESX Nodes, i will start to slowly upgrade my web nodes up to 64bit.
Update:
I have since read on the 'Tubes' that this issue has been resolved in Apache 2.2.x... the server that was having this problem was Apache 2.0.x