takino
/
ordigi
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add support for multiple levels of folders. #199 (#205) 

* Use original name from EXIF instead of parsing assumed file name format. #107
* Updates to docs and code
This commit is contained in:
Jaisen Mathai 2017-03-30 19:43:34 +05:30 committed by GitHub
parent f7be8f323f
commit 74d8675b20
5 changed files with 182 additions and 72 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

71
Readme.md
View File

@ -122,7 +122,8 @@ I work tirelessly to make sure your photos are always sorted and organized so yo
You don't love me yet but you will.
I only do 3 things.
* Firstly I organize your existing collection of photos.
* Firstly I organize your existing collection of photos into a customizable folder structure.
* Second I help make it easy for all the photos you haven't taken yet to flow into the exact location they belong.
* Third but not least I promise to do all this without a yucky propietary database that some friends of mine use.
@ -152,16 +153,15 @@ Updating EXIF of photos from the command line.
I'm most helpful when I'm fully utilized to keep your photos organized.
Here's an example of a very asynchronous setup.
Here's an example of how I can create 3 geographically distributed copies of your meticulously organized photo library.
* Specify a folder in your Dropbox/Google Drive to store the organized photo library.
* Set up a Hazel rule to notify me when photos arrive in `~/Downloads` so I can import them.
* The rule waits 1 minute before processing the photo which gives you a chance to move it elsewhere if it's not something you want in the library.
* Use AirDrop to transfer files from any iPhone to your laptop. That goes to `~/Downloads` for the Hazel rule to process.
* AirDrop is fast, easy for anyone to use and once the transfer is finished your don't have to stick around. I'll move it to Dropbox/Google Drive and Dropbox/Google Drive will sync it to their servers.
* Set up a cron job to import photos in `~/Ready-To-Upload`.
* Add photos to `~/Ready-To-Upload` and wait for your cron job to trigger.
* Periodically recategorize photos by fixing their location or date or by adding them to an album.
* Have a Synology at home set to automatically sync down from Dropbox/Google Drive.
This setup means you can quickly get photos off your or anyone's phone and know that they'll be organized and backed up in 3 locations by the time you're ready to view them.
This setup means you can quickly get photos off your phone or dSLR and know that they'll be organized and backed up in 3 locations by the time you're ready to view or share them.
<p align="center"><img src ="creative/workflow-simplified-white-bg.png" /></p>
@ -209,33 +209,60 @@ OK, so what if you don't like the folders being named `2015-07-Jul/Mountain View
You can add a custom folder structure by editing your `config.ini` file. This is what I include in the sample config file.
#### Custom folder examples
Sometimes examples are easier to understand than explainations so I'll start there. If you'd like to understand my magic I explain it in more detail below these examples. You customize your folder structure in the `Directory` section of your `config.ini`.
```
[Directory]
date=%Y-%m-%b
location=%city
location=%city, %state
year=%Y
full_path=%year/%location
# 2015/Sunnyvale, California
location=%city, %state
month=%B
year=%Y
full_path=%year/%month/%location
# 2015/December/Sunnyvale, California
location=%city, %state
month=%m
year=%Y
date=%year-%month
full_path=%date/%location
# 2015-12/Sunnyvale, California
full_path=%country/%state/%city
# US/California/Sunnyvale
```
There needs to be 2 levels of folders and you can construct them using the date and location. Use `full_path` to determine how the 2 levels are nested. If for some reason your config is not correct I will use the default formatting which is found in `config.ini-sample`.
#### How folder customization works
The default formatting from the above config looks like `2015-07-Jul/Mountain View`.
You can construct your folder structure using a combination of the location and dates. Under the `Directory` section of your `config.ini` file you can define placeholder names and assign each a value. For example, `date=%Y-%m` would create a date placeholder with a value of YYYY-MM which would be filled in with the date from the EXIF on the photo.
#### Customizing the date format
The placeholders can be used to define the folder structure you'd like to create. The example above happens to be the default structure and would look like `2015-07-Jul/Mountain View`.
You can use any of [the standard Python time directives](https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior) to create your ideal structure.
I have a few built-in location placeholders you can use.
* To have `201601`, use `date=%Y%m`
* For `Sunday, 01 January 2016`, use `date=%A, %d %B %Y`
* Python also has some pre-built formats. So you can get `Sun Jan 01 12:34:56 2016`, by using `%c`
* `%city` the name of the city the photo was taken. Requires geolocation data in EXIF.
* `%state` the name of the state the photo was taken. Requires geolocation data in EXIF.
* `%country` the name of the country the photo was taken. Requires geolocation data in EXIF.
#### Customizing the location format
I also have some date placeholders you can customize. You can use any of [the standard Python time directives](https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior) to customize the date format to your liking.
I use the [Open Street Maps Nominatim reverse geocoding API](http://wiki.openstreetmap.org/wiki/Nominatim#Example) provided by MapQuest. You can use `city`, `state` and `country` to construct the folder name.
* `%day` the day the photo was taken.
* `%month` the month the photo was taken.
* `%year` the year the photo was taken.
* To have `Sunnyvale`, use `location=%city`
* To have `Sunnyvale-CA`, use `location=%city-%state
In addition to my built-in and date placeholders you can combine them into a single folder name using my complex placeholders.
Sometimes a location may not have all of the values available. If your format is `%city-%state` and `city` was not returned then the folder name will be `%state`. Take note that I'll strip out extra characters so you don't end up with folders name `-%state` when `city` is not found.
* `%location` can be used to combine multiple values of `%city`, `%state` and `%country`. For example, `location=%city, %state` would result in folder names like `Sunnyvale, California`.
* `%date` can be used to combine multiple values from [the standard Python time directives](https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior). For example, `date=%Y-%m` would result in folder names like `2015-12`.
### Reorganize by changing location and dates

9
elodie/compatability.py
View File

@ -26,8 +26,15 @@ def _decode(string, encoding=sys.getfilesystemencoding()):
def _copyfile(src, dst):
# Python 3 hangs using open/write method
# shutil.copy seems slow, changing to streaming according to
# http://stackoverflow.com/questions/22078621/python-how-to-copy-files-fast # noqa
# Python 3 hangs using open/write method so we proceed with shutil.copy
# and only perform the optimized write for Python 2.
if (constants.python_version == 3):
# Do not use copy2(), it will have an issue when copying to a
# network/mounted drive.
# Using copy and manual set_date_from_filename gets the job done.
# The calling function is responsible for setting the time.
shutil.copy(src, dst)
return

36
elodie/filesystem.py
View File

@ -114,7 +114,7 @@ class FileSystem(object):
# First we check if we have metadata['original_name'].
# We have to do this for backwards compatibility because
# we original did not store this back into EXIF.
if(metadata['original_name'] is not None):
if('original_name' in metadata and metadata['original_name']):
base_name = os.path.splitext(metadata['original_name'])[0]
else:
# If the file has EXIF title we use that in the file name
@ -163,18 +163,18 @@ class FileSystem(object):
config_directory = config['Directory']
path_parts = re.search(
'\%([^/]+)\/\%([^/]+)',
# Find all subpatterns of full_path that map to directories.
# I.e. %foo/%bar => ['foo', 'bar']
path_parts = re.findall(
'\%([a-z]+)',
config_directory['full_path']
)
if not path_parts or len(path_parts.groups()) != 2:
if not path_parts or len(path_parts) == 0:
return self.default_folder_path_definition
path_part_groups = path_parts.groups()
self.cached_folder_path_definition = [
(path_part_groups[0], config_directory[path_part_groups[0]]),
(path_part_groups[1], config_directory[path_part_groups[1]]),
(part, config_directory[part]) for part in path_parts
]
return self.cached_folder_path_definition
@ -188,18 +188,14 @@ class FileSystem(object):
path = []
for path_part in path_parts:
part, mask = path_part
if part == 'date':
if part in ('date', 'day', 'month', 'year'):
path.append(time.strftime(mask, metadata['date_taken']))
elif part == 'location':
if(
metadata['latitude'] is not None and
metadata['longitude'] is not None
):
elif part in ('location', 'city', 'state', 'country'):
place_name = geolocation.place_name(
metadata['latitude'],
metadata['longitude']
)
if(place_name is not None):
location_parts = re.findall('(%[^%]+)', mask)
parsed_folder_name = self.parse_mask_for_location(
mask,
@ -217,11 +213,6 @@ class FileSystem(object):
elif(len(path) == 2):
path[1] = metadata['album']
# if we don't have a 2nd level directory we use 'Unknown Location'
if(len(path) < 2):
path.append('Unknown Location')
# return '/'.join(path[::-1])
return os.path.join(*path)
def parse_mask_for_location(self, mask, location_parts, place_name):
@ -338,11 +329,6 @@ class FileSystem(object):
shutil.move(_file, dest_path)
os.utime(dest_path, (stat.st_atime, stat.st_mtime))
else:
# Do not use copy2(), will have an issue when copying to a
# network/mounted drive using copy and manual
# set_date_from_filename gets the job done
# shutil.copy seems slow, changing to streaming according to
# http://stackoverflow.com/questions/22078621/python-how-to-copy-files-fast # noqa
compatability._copyfile(_file, dest_path)
self.set_utime(media)
@ -352,7 +338,7 @@ class FileSystem(object):
return dest_path
def set_utime(self, media):
""" Set the modification time on the file base on the file name.
""" Set the modification time on the file based on the file name.
"""
# Initialize date taken to what's returned from the metadata function.

14
elodie/geolocation.py
View File

@ -19,6 +19,7 @@ from elodie import log
from elodie.localstorage import Db
__KEY__ = None
__DEFAULT_LOCATION__ = 'Unknown Location'
def coordinates_by_name(name):
@ -115,10 +116,14 @@ def get_key():
def place_name(lat, lon):
lookup_place_name_default = {'default': __DEFAULT_LOCATION__}
if(lat is None or lon is None):
return lookup_place_name_default
# Convert lat/lon to floats
if not isinstance(lat, float):
if(not isinstance(lat, float)):
lat = float(lat)
if not isinstance(lon, float):
if(not isinstance(lon, float)):
lon = float(lon)
# Try to get cached location first
@ -132,8 +137,7 @@ def place_name(lat, lon):
lookup_place_name = {}
geolocation_info = lookup(lat=lat, lon=lon)
if(geolocation_info is not None):
if('address' in geolocation_info):
if(geolocation_info is not None and 'address' in geolocation_info):
address = geolocation_info['address']
for loc in ['city', 'state', 'country']:
if(loc in address):
@ -144,7 +148,7 @@ def place_name(lat, lon):
lookup_place_name['default'] = address[loc]
if('default' not in lookup_place_name):
lookup_place_name = {'default': 'Unknown Location'}
lookup_place_name = lookup_place_name_default
if(lookup_place_name is not {}):
db.add_location(lat, lon, lookup_place_name)

86
elodie/tests/filesystem_test.py
View File

@ -248,6 +248,48 @@ full_path=%date/%location
assert path == os.path.join('2015-12-05','United States of America-California-Sunnyvale'), path
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
def test_get_folder_path_with_with_more_than_two_levels():
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
f.write("""
[Directory]
year=%Y
month=%m
location=%city, %state
full_path=%year/%month/%location
""")
if hasattr(load_config, 'config'):
del load_config.config
filesystem = FileSystem()
media = Photo(helper.get_file('with-location.jpg'))
path = filesystem.get_folder_path(media.get_metadata())
if hasattr(load_config, 'config'):
del load_config.config
assert path == os.path.join('2015','12','Sunnyvale, California'), path
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
def test_get_folder_path_with_with_only_one_level():
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
f.write("""
[Directory]
year=%Y
full_path=%year
""")
if hasattr(load_config, 'config'):
del load_config.config
filesystem = FileSystem()
media = Photo(helper.get_file('plain.jpg'))
path = filesystem.get_folder_path(media.get_metadata())
if hasattr(load_config, 'config'):
del load_config.config
assert path == os.path.join('2015'), path
def test_get_folder_path_with_location_and_title():
filesystem = FileSystem()
media = Photo(helper.get_file('with-location-and-title.jpg'))
@ -658,3 +700,47 @@ full_path=%date/%location
]
if hasattr(load_config, 'config'):
del load_config.config
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
def test_get_folder_path_definition_with_more_than_two_levels():
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
f.write("""
[Directory]
year=%Y
month=%m
day=%d
full_path=%year/%month/%day
""")
if hasattr(load_config, 'config'):
del load_config.config
filesystem = FileSystem()
path_definition = filesystem.get_folder_path_definition()
expected = [
('year', '%Y'), ('month', '%m'), ('day', '%d')
]
if hasattr(load_config, 'config'):
del load_config.config
assert path_definition == expected, path_definition
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
def test_get_folder_path_definition_with_only_one_level():
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
f.write("""
[Directory]
year=%Y
full_path=%year
""")
if hasattr(load_config, 'config'):
del load_config.config
filesystem = FileSystem()
path_definition = filesystem.get_folder_path_definition()
expected = [
('year', '%Y')
]
if hasattr(load_config, 'config'):
del load_config.config
assert path_definition == expected, path_definition