MythicalLibrarian
--this page is under construction
Newest version was released --Outleradam 00:54, 8 January 2010 (UTC)
The Missing Symlink
mythicalLibrarian will convert your recordings into a Video Library, free of commercials, automatically after recording. You can see an example to the right.
About The Program
mythicalLibrarian is a highly versitile tool designed to assign season and episode numbers to television shows based on show title and episode title so they are easily recognized by XBMC or Boxee. mythicalLibrarian interfaces with TheTvDb to gain information about the target file. mythicalLibrarian also has the ability to interface with MythTV's local database, XBMC's command interface, and Ubuntu GNOME desktop to provide an all-around recording recognition/managing engine which will run in the background of your back-end computer to update your library and provide status updates upon each completed job.
mythicalLibrarian pulls information supplied by your MythTV program guide for episodes, and makes a fuzzy logic match out of the supplied data. If fuzzy logic is not enough, it will match original air date. For Movies, mythicalLibrarian will rename your MythTV recordings based upon information obtained from the program guide database on your computer. If you allow commercial data to be generated for your shows, mythicalLibrarian will also copy this data with the recordings. mythicalLibrarian allows MythTV to remain in control of the files, so there is no reason not to process your recordings for boxee and XBMC.
MythicalLibrarian was designed to run under Ubuntu Karmic Koala OS through BASH 4.0
How it works
mythicalLibrarian will generate several files of it's own while running. It must have it's own working directory. To the right, you can see an example of the layout of /mythicalLibrarian, the files which it creates and their purposes. Here is the basic flow:
- You put in- 'mythicalLibrarian.sh "episode or movie name" "episode name" "file/to/process.ext"'
- Episodes- mythicalLibrarian references TheTvDb to gain SeriesID. Get current time from TheTvDb If the database is out-of-date or non existant, it will download the full series and parse it. Then, using fuzzy logic, it makes a match of the series name and gains season and episode information. If this is not possible, it will attempt a match of original airdate from your MythTV program guide database.
mythicalLibrarian will rename your recognizable episodes to: "/Show Name/Show Name.SxxExx (episode title).ext"
- Movies- mythicalLibrarian references your MythTV program guide database to verify it is a movie and then obtains the year for the movie.
mythicalLibrarian will rename your movies to: "/Movie Name/movie title (Year).ext"
Additional Information
- Although mythicalLibrarian has no known issues at the present time, mythicalLibrarian is in beta until it goes one month without problem reports. There is a problem in the fuzzy logic "agrep" pacakage. While not fatal, it causes incorrect recognition of episode number in approximately 1 out of 400 episodes. Please report problems here in the XBMC Forum thread: http://forum.xbmc.org/showthread.php?p=470402#post470402
- If you would like to see additional features and functions, additional back-ends, or additional guide data, please post in this thread http://forum.xbmc.org/showthread.php?t=65769
Installation Setup and Use of mythicalLibrarian
This section covers installation, setup, and use of mythicalLibrarian. After completing this section, mythicalLibrarian can run autonomously without user intervention. See Librarian Interfaces for information on notifications and automation tasks.
Installation
This section covers installation of mythicalLibrarian.
Manual Installation
This section covers manual installation
- Install dependencies
sudo apt-get install curl agrep libnotify-bin
dependencies breakdown:
- curl downloads web pages as text.
- agrep provides fuzzy logic.
- libnotify-bin provides GNOME desktop notifications
- Download the current version of mythcalLibrarian from this download link
- Place mythicalLibrarian.sh in /home/mythtv/mythicalLibrarian
mkdir /home/mythtv/mythicalLibrarian cp /home/yourusername/Downloads/mythicalLibrarian.sh /home/mythtv/mythicalLibrarian
- Make mythicalLibrarian executable by typing the following:
sudo chmod +x /home/mythtv/mythicalLibrarian
This section covers basic setup of mythicalLibrarian. To make mythicalLibrarian work, you will need to set the following settings located in text format within mythicalLibrarian.sh.You can see a picture of the settings to the right.
- MoveDir - this is your primary episode move dir.
- AlternateMoveDir- this is the folder where episodes will be moved if movedir cannot be used
- FailSafeDir- folder to place links if files cannot be moved
- PrimaryMovieDir-Folder to place movies
- AlternateMovieDir- this is where movies will be placed if PrimaryMovieDir fails
- Database username and password should be mythtv/mythtv. When installing MythTV a root user and password was assigned by the user.
Leave the rest as default
Automatic Installation
This section covers automatic installation and upgrade. Automatic upgrade will upgrade you to the SVN version of mythicalLibrarian There may be installation difficulties.
- type the following:
sudo apt-get install curl mkdir ~/.mythicalLibrarian && mkdir ~/.mythicalLibrarian/mythicalSetup && cd ~/.mythicalLibrarian/mythicalSetup curl http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh>./mythicalSetup.sh sudo chmod +x ./mythicalSetup.sh && sudo ./mythicalSetup.sh
- mythicalSetup will run you through a series of prompts and perform the setup of mythicalLibrarian and all of it's features
Automatic Upgrade
This section covers upgrading of mythicalLibrarian
- type the following
sudo mythicalLibrarian --update
- mythicalLibrarian will download the newest version of mythicalSetup and perform the upgrade
Use of mythicalLibrarian on MythTV Applications
This section covers setup of MythTV without any changes to mythicalLibrarian settings. This will create only a basic setup. Make sure to follow up by checking the /home/mythtv/mythicalLibrarian/output.log file after the first run.
- Download mythicalLibrarian.sh and put it on your desktop: http://mythicallibrarian.googlecode.com/files/mythicalLibrarian.sh
- Download librarian-notify-send and put it on your desktop: http://mythicallibrarian.googlecode.com/files/librarian-notify-send
- Type the following into a terminal
sudo apt-get install curl agrep libnotify-bin sudo mkdir /home/mythtv /home/mythtv/mythicalLibrarian /home/mythtv/Episodes /home/mythtv/Movies /home/mythtv/Videos /home/mythtv/FailSafe sudo chown -R mythtv:mythtv /home/mythtv /home/mythtv/mythicalLibrarian /home/mythtv/Episodes /home/mythtv/Movies /home/mythtv/Videos /home/mythtv/FailSafe sudo mv ~/Desktop/mythicalLibrarian.sh /home/mythtv/mythicalLibrarian/mythicalLibrarian.sh sudo mv ~/Desktop/librarian-notify-send /usr/local/bin/librarian-notify-send sudo chmod +x /usr/local/bin/librarian-notify-send sudo chmod +x /home/mythtv/mythicalLibrarian/mythicalLibrarian.sh
MythTV settings
See use of mythicalLibrarian on MythTV Applications for setting up mythtv here: http://wiki.xbmc.org/index.php?title=MythicalLibrarian&action=edit§ion=4
sudo adduser *Your User Name here* mythtv
- Click System/Administration MythTV Baclend setup
- Select "1. General" then select next until you get to screen 3
- Check "Follow symbolic links when deleting files".
- Select next until you get to screen 8
- Check "Allow user job #1 Jobs"
- Select next until you get to screen 10
UserJob #1 Description: mythicalLibrarian User Job #1 Command: /usr/local/bin/mythicalLibrarian "%DIR%/%FILE%"
- Select Finish to exit
- Select 6. Storage Directories
- Select Default
- Add a recording folder to which MythTV has access ie. /home/mythtv/Videos
- Add additional folders if required.
- Remove folders to which MythTV cannot run move jobs such as /lib or /var
Use of mythicalLibrarian on non-MythTV Applications
This section covers setup of non-MythTV applications. Make sure permissions are set correctly.
- edit mythicalLibrarian.sh and set Database=Disabled
- Set mythicalLibrarian up to run the job as follows:
/home/mythtv/mythicalLibrarian/mythicalLibrarian.sh "Show name" "Show Title" "File to rename"
- see Setting the Settings in this section
Using the command line
mythicalLibrarian has many command line switches. This section covers command line use of mythicalLibrarian.
mythicalLibrarian Normal Operation
This section covers normal operation of mythicalLibrarian When using mythicalLibrarian to rename files individually without database access, you can run the following command. Without database access, mythicalLibrarian can only look up episodes with title and subtitle.
mythicalLibrarian /home/mythtv/videos/1024_33020100321.mpg "spongebob squarepants" "giant squidward" mythicalLibrarian /path/to/video/file.ext "episode title" "episode subtitle"
When using mythicalLibrarian with the mythtv database access you can run mythicalLibrarian as the following command.
mythicalLibrarian /home/mythtv/videos/1024_33020100321.mpg mythicalLibrarian /path/to/mythtv/videos/filename.ext
mythicalLibrarian will integrate with the mythtv database as a user job after recordings. The mythtv user job for this is:
/usr/local/bin/mythicalLibrarian "%DIR%/%FILE%"
mythicalLibrarian --help
This section covers -h|--help
mythicalLibrarian --help
--help will display a help message containing helpful information
mythicalLibrarian --scan
This section covers -s|--scan
mythicalLibrarian --scan mpg /home/mythtv/Videos mythicalLibrarian --scan ext /path/to/video/files
--scan will search for files automatically and process them. --scan has two parmaters. The first parameter is the extension of the files to search. The seccond parameter is the folder to search.
mythicalLibrarian --doover
This section covers -d|--doover
mythicalLibrarian --doover
mythicalLibrarian keeps track of each file which fails while it runs. At any time you can type "mythicalLibrarian --doover" and all previously failed files will be processed. This process cannot be interrupted for any reason or data loss may occour.
mythicalLibrarian --undo
This section covers -u|--undo
mythicalLibrarian --undo
mythicalLibrarian keeps track of every file which has been moved. At any time you can type mythicalLibrarian --undo and all files will be reverted to their original state. There is no indication that --doover is running until it completes. This process cannot be interrupted or data loss may occour.
mythicalLibrarian --update
This section covers --update
sudo mythicalLibrarian --update
--update is a complicated process. First, mythicalLibrarian will download a copy of the mythicalSetup from the SVN repository. Next, it will download a copy of mythicalLibrarian from the SVN repository. mythicalSetup will then separate the program from the user settings. mythicalSetup perform several checks and bring up a GUI. The GUI will ask the user for preferences. mythicalSetup will attempt to set permissions and settings to appropriate defaults based upon user settings and the system. mythicalSetup will then combine the user settings with the separated program and place the file in /usr/local/bin. mythicalSetup will then call mythicalLibrarian --mythicalDiagnostics, which checks out your system. mythicalDiagnostics then calls mythcalLibrarian testfile.ext "mister" "episode 1332", which puts mythicalLibrarian into a diagnostic run. Once complete, the mythicalLibrarian in test mode, mythicalDiagnostics, mythicalSetup and update are closed and results are returned to the user.
mythicalLibrarian --mythicalDiagnostics
This section covers -m|--mythicalDiagnostics
mythcalLibrarian --mythicalDiagnostics
--mythicalDiagnostics is a step-by-step functions check of mythicalLibrarian which will test functionality. It tests for all known causes of failure. This is a good troubleshooting tool for you to work the kinks out of your settings. mythicalLibrarian runs a series of diagnostics and then runs a search for an episode of mr rodgers with the file name set to testfile.ext. testfile.ext truncates the execution of mythicalLibrarian before any file operations occur. mythicalDiagnostics will return results which should look like this
All tests complete RESULTS Installed Packages: ***PASSED*** Notifications Test: ***PASSED*** XBMC Communications: ***PASSED*** File System Checks: ***PASSED*** mythicalTesting: ***PASSED*** Overall mythicalReadiness: ***PASSED*** -----End of diagnostics-----
Notifications test and xbmc communications test will not fail overall mythicalReadiness. Only critical checks effect overall mythicalReadiness. Critical checks are dependencies, file system, and mythicalTesting.
Librarian Interfaces
mythicalLibrarian has the capability to send notifications to the Ubuntu GNOME desktop and keep your XBMC library up-to-date and clean. This section covers setup of notifications of the various platforms supported by mythicalLibrarian
XBMC Commands
This section covers setup of XBMC Commands. mythicalLibrarain can keep XBMC's library up to date, clean out deleted files and send a notification upon a completed job.
- edit mythicalLibrarian.sh in your text editor
- set XBMCUpdate, XBMCClean, and XBMCNotify to the desired settings, either Enabled, or Disabled
- Set the IP and port under XBMCIPs. Separate IP addresses with a space. eg.
XBMCIPs=( "192.168.1.110:8080 192.168.1.111:8080 XBOX:8080" )
GNOME Desktop Notifications
- mythicalLibrarian requires no-password sudo access to send GNOME notifications.
GNOME Desktop notifications can be turned on and off from within the script
GNOME Desktop notifications are used on the back-end machine by mythicalLibrarian to let you know that a file move operation is in process or an operation has completed. The external script librarian-notify-send becomes the specified username and sends a GNOME desktop notification. librarian-notify-send requires the user mythtv to have access to the sudo command without a password if run as a user job.
- Get the current version of librarian-notify-send from Google Code download link
- place the file in /usr/local/bin
- add mythtvall group
sudo addgroup mythtvall
- add mythtv to the mythtvall group
sudo adduser mythtv mythtvall
- edit the sudoers file
sudo visudo
- add the following line to the sudoers
%mythtvall ALL=(ALL) NOPASSWD:ALL
- hit ctrl-x and then y, then enter to write out and quit visudo
- make librarian-notify-send executable by typing
sudo chmod +x /usr/local/bin/librarian-notify-send
- edit mythicalLibrarian and set Notify=Enabled and NotifyUserName=*your user name*
You are done.
Troubleshooting
While mythicalLibrarian is designed to run fully autonomously, configuration issues come up from time to time. This section deals with these set-up issues
Random Errors
This section of Troubleshooting is designated for random errors which make no sense. These include errors which make you think mythicalLibrarian simply does not work.
Dependencies
This section covers dependencies of mythicalLibrarian. Usually you can find dependencies by typing "command -V" or "command --version"
- The following are the versions of the binaries which mythicalLibrarian depends upon and are not in OS Karmic Koala.
agrep -V This is agrep version 3.0, 1994. curl 7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.15 dialog Version: 1.1-20080819 mysql Ver 14.14 Distrib 5.1.37, for debian-linux-gnu (i486) using EditLine wrapper notify-send 0.4.5
- The Following are the versions of the binaries which mythicalLibrarian depends upon and are included in OS Karmic Koala.
bash GNU bash, version 4.0.33(1)-release (i486-pc-linux-gnu) (GNU coreutils) 7.4 sed GNU sed version 4.2.1 grep GNU grep 2.5.4 awk GNU Awk 3.1.6 ifconfig 1.42 (2001-04-13) net-tools 1.60 replace Ver 1.4 for debian-linux-gnu at i486
Permission Errors
This section of Troubleshooting is designated for permission errors.
permission errors when run from terminal
Ensure the user running mythicalLibrarian has permission to run mythicalLibrarian. This includes read and write permissions to the working folder (default=~/mythicalLibrarian) see this post for more information: http://forum.xbmc.org/showpost.php?p=489246&postcount=36
mythicalLibrarian is not generating a log
Make sure mythicalLibrarian.sh is executable. You can set the permissions by typing "sudo chmod +x /path/to/mythicalLibrarian.sh" note: mythtv will be the user running mythicalLibrarian when run as a user job from MythTV see this post for more information: http://forum.xbmc.org/showpost.php?p=489246&postcount=36
Files are not being renamed or moved, but they are owned my mythtv
mythtv must have full access to the files. the /var/ folder does not allow users to move files without special permissions. Try moving your recording directory to your /home/ folder
mythicalLibrarian works from terminal, but not from user job
"mythtv" is the user calling the user job. If the user running mythicalLibrarian does not have access to the file, then mythicalLibrarian will not have access to the file. MythTV users can troubleshoot by changing your username with "sudo su mythtv", then running mythicalLibrarian.
mythcialLibrarian will not copy to my NFTF/FAT/EXT3 mounted drive
Additonal file systems may require editing the /etc/fstab file to change permissions. Search your OS documentation for information on how to mount a file system under a different username. Alternately, you may wish to add the user running mythicalLibrarian to your user group.
XBMC library is not updated
This section of Troubleshooting is designated for XBMC specific issues
XBMC does not show notifications
mythicalLibrarian requires the user to set the IP address of XBMC within mythicalLibrarian.sh
- ensure you have a static IP set up in your router, see your router's manufacturer's website for more information
- Open mythicalLibrarian.sh and edit the "XBMCIPs" setting to your XBMC IP.
XBMC shows notifications but Library is not updated
- Add a source http://wiki.xbmc.org/?title=Adding_Media_Sources
- Set the content of your video library http://wiki.xbmc.org/?title=Set_Contents_%28Video_Library%29
OS Speciffic issues
This section of Troubleshooting is designated for OS Specific issues
Gentoo
Gentoo OS requires upgrade of many packages. For sure you will need the package TRE. See here for version information: http://forum.xbmc.org/showpost.php?p=495697&postcount=62
Problems after upgrade to new version
This section of Troubleshooting is designated for issues after upgrade when the previous version was working
Episodes are not identifed
When episodes are not identified after an upgrade it is because the database handling has been upgraded, however, the database itself has not. Delete all database folders to correct the issue http://forum.xbmc.org/showpost.php?p=492001&postcount=40
mythicalLibrarian does not generate any files in it's working directory
Ensure the user running mythicalLibrarian has permission to run mythicalLibrarian. This includes read and write permissions to the working folder (default=~/mythicalLibrarian)
Episode recognition problems
Series title is not properly recognized
TheTvDb.com does not have the proper Zap2it id included. Edit it. The program guide uses a generic name. Apply a showtranslation. See documentation included within mythicalLibrarian.sh.
Episode title was not properly recognized
TheTvDb.com is down. Wait for TheTvDb.com to resume and then run the doover.sh file generated by mythicalLibrarian Ensure the data at TheTvDb is accurate. Correct if needed.
Gnome Desktop Notifications
This section of Troubleshooting is designated for GNOME Desktop Notifications issues
Gnome Desktop Notifications are not showing up
Ensure you have Notify=Enabled set in mythicalLibrarian.sh See this section for more information http://wiki.xbmc.org/index.php?title=MythicalLibrarian&action=submit#GNOME_Desktop_Notifications
Guide Data errors
for SchedulesDirect/Tribune/Zap2it set the GuideDataType setting in mythicalLibrarian.sh to "SchedulesDirect"
Currently Unsupported Guide data types
Support for guide data will be available with help from you. See this post: http://forum.xbmc.org/showpost.php?p=496721&postcount=90
Upgrade Issues
This section deals with issues relating to upgrading mythicalLibrarian
COMSKIP files are not being deleted
The comskip tracking file has been upgraded to be an overall file tracking system where all created files are linked to a single master file. When the master file is deleted manually or because mythtv required extra free space, the associated files will be deleted on the next run of mythicalLibrarian. See this post for the upgrade tool. http://forum.xbmc.org/showpost.php?p=507770&postcount=209
Notes
mythicalLibrarian is a spawn from the project BashSExx and MythSExx.
MythSExx can be used if you are strictly trying to rename episodes
mythicalLibrarian is currently in advanced beta stage.