Ripping off porn sites (how-to)

What you will need:

• Unix Any clone will do, such as GNU/Linux, BSD or OS X.
• curl This comes with many unixes by default.
• wget You'll probably have to download this one.

Method:

This will work with any sequentially numbered files, but pornographic pictures are the most obvious example. Only one of curl and wget is required, although having both installed is a bonus.

• Find a site offering free files There are presumably many sites that have daily updated lists of links to free preview galleries. Find one, and check out the sites it links to until you find one that has a picture in a location such as /free_preview/gallery/photo_shoot_023/016.jpg.

What you do next, and which tools you use to do it, depends on exactly how many files you want to leech, and how neatly you want them stored on your filesystem.

• Leech the pictures or movies on a single page You can do this using wget with a simple one-liner: wget -r -N -A.jpg -R_thumbnail.jpg http://www.website-name.com/free_preview_gallery/photo_shoot_023/index.html Change the first .jpg to whichever file format you want to download, and change _thumbnail.jpg to whatever the thumbnail pictures end with, so that those aren't downloaded. wget will cheerfully download all the full-sized images linked from that page. This works regardless of the filenames, so it's your best bet if the filenames aren't sequential numbers.
• Leech sequentially numbered files This is more of curl's forte. If you're lucky enough to find a picture called something like 016.jpg, use trial and error to find the lowest number (usually 00, 01, 0 or 1) and highest number (usually around 15 or 20, or thereabouts). Then simply use curl -O http://www.website-name.com/free_preview_gallery/photo_shoot_023/[01-20].jpg The good thing about curl is it works regardless of if you include trailing zeroes or not, as long as you match the filename on the site. The disadvantage, however, is that it won't preserve the directory structure of the web site; it will just save the files to your current directory. Note that that's a capital letter "o" at the beginning, not a zero.
• Leech sequentially numbered files within sequentially numbered directories This is where the real automated fun begins: rather than just download a single photo shoot, you can download them all. Use trial and error again, this time to find the lowest and highest number in the directory names. This only works if every directory has the exact same number of files in it. This time, type in the slightly more complex line: curl http://www.website-name.com/free_preview_gallery/photo_shoot_[001-049]/[01-20].jpg -o website-name_#1_#2.jpg This works in a similar way to before, only the -o (lower case letter "o") tells curl that you want it to save the files with different names. If you didn't do this, it would leech every single file, but overwrite each set of twenty photos with the next one.
• Leech sequentially numbered files within sequentially numbered directories, preserving the directory names and filenames Preserving the site's directory structure is something that wget is best at. Unfortunately, wget can't count like curl can, so you'll need to write a small shell script to do that part:

#!/bin/sh

count_a=1
while [ $count_a -le 9 ]
do
  count_b=1
  while [ $count_b -le 9 ]
  do
    echo "Downloading batch $count_a.0$count_b of 49.20"
    wget -q -x -N http://www.website-name.com/free_preview_gallery/photo_shoot_00$count_a/0$count_b.jpg
    count_b=`expr $count_b + 1`
  done
  count_b=10
  while [ $count_b -le 20 ]
  do
    echo "Downloading batch $count_a.$count_b of 49.20"
    wget -q -x -N http://www.website-name.com/free_preview_gallery/photo_shoot_00$count_a/$count_b.jpg
    count_b=`expr $count_b + 1`
  done
  count_a=`expr $count_a + 1`
done
count_a=10
while [ $count_a -le 49 ]
do
  count_b=1
  while [ $count_b -le 9 ]
  do
    echo "Downloading batch $count_a.0$count_b of 49.20"
    wget -q -x -N http://www.website-name.com/free_preview_gallery/photo_shoot_0$count_a/0$count_b.jpg
    count_b=`expr $count_b + 1`
  done
  count_b=10
  while [ $count_b -le 20 ]
  do
    echo "Downloading batch $count_a.$count_b of 49.20"
    wget -q -x -N http://www.website-name.com/free_preview_gallery/photo_shoot_0$count_a/$count_b.jpg
    count_b=`expr $count_b + 1`
  done
  count_a=`expr $count_a + 1`
done

If you're not familiar with scripts, just remember to chmod 755 the file and type in ./script.sh to run it, where script.sh is the filename you save it as. Make sure #!/bin/sh points to the correct location of the shell you want to use by using whereis.

As you can see from the script, counting this way only works without starting with a preceding zero, so using curl is much easier for leeching from multiple directories this way.

Of course, wget is at its best when you let it leech an entire site, in which case it'll work out all of the directory names and filenames by just following the links, but that would put a major strain on the server. You wouldn't want to do that, would you?