How to fix Bashpodder / wget and Libsyn file extensions

 

  Those who have had very long jiberish file extensions attached to the files downloaded from libsyn my be interested in this information. Recently libsyn added some extra information to their files to control how the downloads are handled.I found that the conflict is with “wget” which is used by many of the linux/unix podcast clients from what I understand.  I communicated with libsyn via email a few times and they were very helpful in trying to find a solution to the problem. The program I use is Bashpodder which of course uses wget. I have found it to be an amazing utility for downloading my list of many podcasts until recently. I am not interested in trying several different podcast clients. I just want Bashpodder to work again.

 

   Within some of the emails from libsyn I was sent the code to a script from the tech team which used the curl application. It did work fine for single downloads of podcasts from libsyn but I got it to work with Bashpodder it seemed to break all of my other podcast downloads. I spent most of my spare time the next few days trying to figure it out and now I have a solution. I was able to find the part of the Bashpodder script that called wget and changed it to point to a script that I wrote to take care of the actual downloads. I use a modified version of Bashpodder so it took some time to make this work but looking back now, it wasn't that hard.

 

   Here is a copy of the Bashpodder script that I use.

 


#!/bin/bash

# By Linc 10/1/2004

# Find the latest script at http://linc.homeunix.org:8080/scripts/bashpodder

# If you use this and have made improvements or have comments

# drop me an email at linc dot fessenden at gmail dot com

# I'd appreciate it!

#

# This revision by Brian Hefferan 2004/02/06, adding configuration options.

# No warranty. It seems to work for me, I hope it works for you.

# Questions /corrections on the additions by Brian Hefferan can be sent to

# brian at heftone  dot  com



#default values can be set here. Command-line flags override theses.

verbose=true

#wget_quiet='-q'  #default is -q

wget_quiet='--tries=1 -q'  #default is -q

wget_continue=

catchup_all=

first_only=

unix2dos=

usetorrents=

sync_disks=

fetchlist='bp.conf'







function usage

{

  echo "

Usage: $0 [OPTIONS]

Options are:

-v, --verbose          display verbose messages. Also enables wget's continue

                      option.

--catchup_all          write all urls to the log file without downloading the

                      actual podcasts. This is useful if you want to subscribe

                      to some podcasts but don't want to download all the back

                      issues. You can edit the podcast.log file afterwards to

                      delete any url you still wish to download next time

                      bashpodder is run.

--first_only           grab only the first new enclosed file found in each feed.

                      The --catchup_all flag won't work with this option. If

                      you want to download the first file and also permanently

                      ignore the other files, run bashpodder with this option,

                      and then run it again with --catchup_all.

-bt --bittorrent       launch bittorrent for any .torrent files downloaded.

                      Bittorrent must be installed for this to work. The

                      the script and bittorrent process will continue running

                      in the foreground indefinitely. You can use ctr-c to

                      kill it when you want to stop participating in the

                      torrent.

--sync_disks           run the "sync" command twice when finished. This helps

                      makes sure all data is written to disk. Recommended if

                      data is being written directly to a portable player or

                      other removable media.

-u, --url_list         ignore bp.conf, instead use url(s) provided on the

                      command line. The urls should point to rss feeds.

                      If used, this needs to be the last option on the

                      command line. This can be used to quickly download just

                      a favorite podcast, or to take a few new podcasts for a

                      trial spin.

-h, --help             display this help message



"

}



if [ -n "$verbose" ]; then wget_quiet='';wget_continue='-c';fi

if test -f urls.temp;then rm urls.temp;fi



# Make script crontab friendly:

cd $(dirname $0)



while [ "$1" != "" ];do

   case $1 in

             -v|--verbose ) verbose=1

                            wget_continue='-c'

                            wget_quiet=''

                         ;;

            -u|--url_list ) shift

                            while [ "$1" != "" ];do

                               echo "$1" >> urls.temp

                               shift

                            done

                            if test ! -f urls.temp

                               then

                                   echo "Error: -u or --url_list option specified, but no urls given on command line. quitting."

                                   exit 1;

                            fi

                            fetchlist='urls.temp'

                         ;;

            --catchup_all ) catchup_all=1

                         ;;

             --first_only ) first_only=1

                         ;;

             --bittorrent ) usetorrents=1

                         ;;

             --sync_disks ) sync_disks=1

                         ;;

                -h|--help ) usage

                            exit

                         ;;

   esac

   shift

done



# datadir is the directory you want podcasts saved to:

datadir=$(date +%Y-%m-%d)



# Check for and create datadir if necessary:

if test ! -d $datadir

      then

      mkdir $datadir

fi



if test ! -f bp.conf && test ! -f urls.temp;

then

   echo "Sorry no bp.conf found, and no urls in command line. Run $0 -h for usage."

   exit

fi



# Read the bp.conf file and wget any url not already in the podcast.log file:

while read podcast

      do

# Skip lines beginning with '#' as comment lines - from Rick Slater

       if echo $podcast | grep '^#' > /dev/null

               then

               continue

       fi


      seenfirst=

      if [ -n "$verbose" ]; then echo "fetching rss $podcast...";fi;

      for url in $(wget -q "$podcast" -O - | tr '\r' '\n' | tr \' \" | \

                   sed -n 's/.*url *= *"\([^"]*\)".*/\1/p' )

              do

          if [ -n "$first_only" ] && [ -n "$seenfirst" ]; then break;fi

          echo $url >> temp.log

          if [ -n "$catchup_all" ];

          then

              if [ -n "$verbose" ]; then echo " catching up $url...";fi

          elif   ! grep "$url" podcast.log > /dev/null ;

          then

             if [ -n "$verbose" ]; then echo "  downloading $url...";fi

             myscript "$url" $wget_continue $wget_quiet -P $datadir "$url"

          fi

          seenfirst=1

      done

done < $fetchlist



if test ! -f temp.log && [ -n "$verbose" ];then echo "nothing to download.";fi



if test -f urls.temp; then rm urls.temp;fi



# Move dynamically created log file to permanent log file:

cat podcast.log >> temp.log

sort temp.log | uniq > podcast.log

rm temp.log



# Use bittorrent to download any files pointed from bittorrent files:

if [ "$usetorrents" ] 

then

    if ls $datadir/*.torrent 2> /dev/null

    then

          btlaunchmany.py $datadir

    fi

fi



# Create an m3u playlist:

ls -1rc $datadir | grep -v m3u > $datadir/podcast${datadir}.m3u

if [ -n "$unix2dos" ];then unix2dos $datadir/podcast${datadir}.m3u;fi;



if [ -n "$sync_disks" ]

then

    if [ -n "$verbose" ]; then echo "running sync..";fi;

    sync

    if [ -n "$verbose" ]; then echo "running sync again..";fi;

    sync

fi



if [ -n "$verbose" ]; then echo "done.";fi;



You will notice the line that says

 

wget $wget_continue $wget_quiet -P $datadir "$url"

 

This is the line that I had to change. It now looks like this

 

myscript "$url" $wget_continue $wget_quiet -P $datadir "$url"

 

 

  This will tell Bashpodder to download with my script using a program called “snarf”.

  If you are using ubuntu you can just install snarf from synaptic or from the command line using

  sudo apt-get install snarf
  

   Here is a copy of the script that I have written.

 

 

#! /bin/bash
URL=$1
datadir=$(date +%Y-%m-%d)

# Check for and create datadir if necessary:
if test ! -d $datadir
      then
      mkdir $datadir
fi

cd $datadir
snarf -v $URL

 Save this to a file and name it whatever you like. Make it executable and copy it to your /usr/bin directory. Or if you have followed my previous post here you can put it in your /home/username/bin directory. Next time you run bashpodder you will have the files downloaded properly and no file extension issues. I hope that this will help the other people that are having the same trouble I have experienced. If anyone has any ideas to make this better or would like to share what they have done to take care of the issue, please let me know – drew at my brain runs linux dot com -

 

 

 UPDATE:

 

I contacted Link of the LinuxLinkTechShow and he informed me that this is an old version of the bashpodder script.

If you upgrade to the new script it will take care of the libsyn problem.

If you insist on using the old version like mine, my solution is still working perfectly.