* Use original name from EXIF instead of parsing assumed file name format. #107 * Updates to docs and code
This commit is contained in:
Jaisen Mathai
GitHub
parent
f7be8f323f
commit
74d8675b20
71
Readme.md
71
Readme.md
|
|
@ -122,7 +122,8 @@ I work tirelessly to make sure your photos are always sorted and organized so yo
|
|||
You don't love me yet but you will.
|
||||
|
||||
I only do 3 things.
|
||||
* Firstly I organize your existing collection of photos.
|
||||
|
||||
* Firstly I organize your existing collection of photos into a customizable folder structure.
|
||||
* Second I help make it easy for all the photos you haven't taken yet to flow into the exact location they belong.
|
||||
* Third but not least I promise to do all this without a yucky propietary database that some friends of mine use.
|
||||
|
||||
|
|
@ -152,16 +153,15 @@ Updating EXIF of photos from the command line.
|
|||
|
||||
I'm most helpful when I'm fully utilized to keep your photos organized.
|
||||
|
||||
Here's an example of a very asynchronous setup.
|
||||
Here's an example of how I can create 3 geographically distributed copies of your meticulously organized photo library.
|
||||
|
||||
* Specify a folder in your Dropbox/Google Drive to store the organized photo library.
|
||||
* Set up a Hazel rule to notify me when photos arrive in `~/Downloads` so I can import them.
|
||||
* The rule waits 1 minute before processing the photo which gives you a chance to move it elsewhere if it's not something you want in the library.
|
||||
* Use AirDrop to transfer files from any iPhone to your laptop. That goes to `~/Downloads` for the Hazel rule to process.
|
||||
* AirDrop is fast, easy for anyone to use and once the transfer is finished your don't have to stick around. I'll move it to Dropbox/Google Drive and Dropbox/Google Drive will sync it to their servers.
|
||||
* Set up a cron job to import photos in `~/Ready-To-Upload`.
|
||||
* Add photos to `~/Ready-To-Upload` and wait for your cron job to trigger.
|
||||
* Periodically recategorize photos by fixing their location or date or by adding them to an album.
|
||||
* Have a Synology at home set to automatically sync down from Dropbox/Google Drive.
|
||||
|
||||
This setup means you can quickly get photos off your or anyone's phone and know that they'll be organized and backed up in 3 locations by the time you're ready to view them.
|
||||
This setup means you can quickly get photos off your phone or dSLR and know that they'll be organized and backed up in 3 locations by the time you're ready to view or share them.
|
||||
|
||||
<p align="center"><img src ="creative/workflow-simplified-white-bg.png" /></p>
|
||||
|
||||
|
|
@ -209,33 +209,60 @@ OK, so what if you don't like the folders being named `2015-07-Jul/Mountain View
|
|||
|
||||
You can add a custom folder structure by editing your `config.ini` file. This is what I include in the sample config file.
|
||||
|
||||
#### Custom folder examples
|
||||
|
||||
Sometimes examples are easier to understand than explainations so I'll start there. If you'd like to understand my magic I explain it in more detail below these examples. You customize your folder structure in the `Directory` section of your `config.ini`.
|
||||
|
||||
```
|
||||
[Directory]
|
||||
date=%Y-%m-%b
|
||||
location=%city
|
||||
location=%city, %state
|
||||
year=%Y
|
||||
full_path=%year/%location
|
||||
|
||||
# 2015/Sunnyvale, California
|
||||
|
||||
location=%city, %state
|
||||
month=%B
|
||||
year=%Y
|
||||
full_path=%year/%month/%location
|
||||
|
||||
# 2015/December/Sunnyvale, California
|
||||
|
||||
location=%city, %state
|
||||
month=%m
|
||||
year=%Y
|
||||
date=%year-%month
|
||||
full_path=%date/%location
|
||||
|
||||
# 2015-12/Sunnyvale, California
|
||||
|
||||
full_path=%country/%state/%city
|
||||
|
||||
# US/California/Sunnyvale
|
||||
|
||||
```
|
||||
|
||||
There needs to be 2 levels of folders and you can construct them using the date and location. Use `full_path` to determine how the 2 levels are nested. If for some reason your config is not correct I will use the default formatting which is found in `config.ini-sample`.
|
||||
#### How folder customization works
|
||||
|
||||
The default formatting from the above config looks like `2015-07-Jul/Mountain View`.
|
||||
You can construct your folder structure using a combination of the location and dates. Under the `Directory` section of your `config.ini` file you can define placeholder names and assign each a value. For example, `date=%Y-%m` would create a date placeholder with a value of YYYY-MM which would be filled in with the date from the EXIF on the photo.
|
||||
|
||||
#### Customizing the date format
|
||||
The placeholders can be used to define the folder structure you'd like to create. The example above happens to be the default structure and would look like `2015-07-Jul/Mountain View`.
|
||||
|
||||
You can use any of [the standard Python time directives](https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior) to create your ideal structure.
|
||||
I have a few built-in location placeholders you can use.
|
||||
|
||||
* To have `201601`, use `date=%Y%m`
|
||||
* For `Sunday, 01 January 2016`, use `date=%A, %d %B %Y`
|
||||
* Python also has some pre-built formats. So you can get `Sun Jan 01 12:34:56 2016`, by using `%c`
|
||||
* `%city` the name of the city the photo was taken. Requires geolocation data in EXIF.
|
||||
* `%state` the name of the state the photo was taken. Requires geolocation data in EXIF.
|
||||
* `%country` the name of the country the photo was taken. Requires geolocation data in EXIF.
|
||||
|
||||
#### Customizing the location format
|
||||
I also have some date placeholders you can customize. You can use any of [the standard Python time directives](https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior) to customize the date format to your liking.
|
||||
|
||||
I use the [Open Street Maps Nominatim reverse geocoding API](http://wiki.openstreetmap.org/wiki/Nominatim#Example) provided by MapQuest. You can use `city`, `state` and `country` to construct the folder name.
|
||||
* `%day` the day the photo was taken.
|
||||
* `%month` the month the photo was taken.
|
||||
* `%year` the year the photo was taken.
|
||||
|
||||
* To have `Sunnyvale`, use `location=%city`
|
||||
* To have `Sunnyvale-CA`, use `location=%city-%state
|
||||
In addition to my built-in and date placeholders you can combine them into a single folder name using my complex placeholders.
|
||||
|
||||
Sometimes a location may not have all of the values available. If your format is `%city-%state` and `city` was not returned then the folder name will be `%state`. Take note that I'll strip out extra characters so you don't end up with folders name `-%state` when `city` is not found.
|
||||
* `%location` can be used to combine multiple values of `%city`, `%state` and `%country`. For example, `location=%city, %state` would result in folder names like `Sunnyvale, California`.
|
||||
* `%date` can be used to combine multiple values from [the standard Python time directives](https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior). For example, `date=%Y-%m` would result in folder names like `2015-12`.
|
||||
|
||||
### Reorganize by changing location and dates
|
||||
|
||||
|
|
|
|||
|
|
@ -26,8 +26,15 @@ def _decode(string, encoding=sys.getfilesystemencoding()):
|
|||
|
||||
|
||||
def _copyfile(src, dst):
|
||||
# Python 3 hangs using open/write method
|
||||
# shutil.copy seems slow, changing to streaming according to
|
||||
# http://stackoverflow.com/questions/22078621/python-how-to-copy-files-fast # noqa
|
||||
# Python 3 hangs using open/write method so we proceed with shutil.copy
|
||||
# and only perform the optimized write for Python 2.
|
||||
if (constants.python_version == 3):
|
||||
# Do not use copy2(), it will have an issue when copying to a
|
||||
# network/mounted drive.
|
||||
# Using copy and manual set_date_from_filename gets the job done.
|
||||
# The calling function is responsible for setting the time.
|
||||
shutil.copy(src, dst)
|
||||
return
|
||||
|
||||
|
|
|
|||
|
|
@ -114,7 +114,7 @@ class FileSystem(object):
|
|||
# First we check if we have metadata['original_name'].
|
||||
# We have to do this for backwards compatibility because
|
||||
# we original did not store this back into EXIF.
|
||||
if(metadata['original_name'] is not None):
|
||||
if('original_name' in metadata and metadata['original_name']):
|
||||
base_name = os.path.splitext(metadata['original_name'])[0]
|
||||
else:
|
||||
# If the file has EXIF title we use that in the file name
|
||||
|
|
@ -163,18 +163,18 @@ class FileSystem(object):
|
|||
|
||||
config_directory = config['Directory']
|
||||
|
||||
path_parts = re.search(
|
||||
'\%([^/]+)\/\%([^/]+)',
|
||||
# Find all subpatterns of full_path that map to directories.
|
||||
# I.e. %foo/%bar => ['foo', 'bar']
|
||||
path_parts = re.findall(
|
||||
'\%([a-z]+)',
|
||||
config_directory['full_path']
|
||||
)
|
||||
|
||||
if not path_parts or len(path_parts.groups()) != 2:
|
||||
if not path_parts or len(path_parts) == 0:
|
||||
return self.default_folder_path_definition
|
||||
|
||||
path_part_groups = path_parts.groups()
|
||||
self.cached_folder_path_definition = [
|
||||
(path_part_groups[0], config_directory[path_part_groups[0]]),
|
||||
(path_part_groups[1], config_directory[path_part_groups[1]]),
|
||||
(part, config_directory[part]) for part in path_parts
|
||||
]
|
||||
return self.cached_folder_path_definition
|
||||
|
||||
|
|
@ -188,25 +188,21 @@ class FileSystem(object):
|
|||
path = []
|
||||
for path_part in path_parts:
|
||||
part, mask = path_part
|
||||
if part == 'date':
|
||||
if part in ('date', 'day', 'month', 'year'):
|
||||
path.append(time.strftime(mask, metadata['date_taken']))
|
||||
elif part == 'location':
|
||||
if(
|
||||
metadata['latitude'] is not None and
|
||||
metadata['longitude'] is not None
|
||||
):
|
||||
place_name = geolocation.place_name(
|
||||
metadata['latitude'],
|
||||
metadata['longitude']
|
||||
)
|
||||
if(place_name is not None):
|
||||
location_parts = re.findall('(%[^%]+)', mask)
|
||||
parsed_folder_name = self.parse_mask_for_location(
|
||||
mask,
|
||||
location_parts,
|
||||
place_name,
|
||||
)
|
||||
path.append(parsed_folder_name)
|
||||
elif part in ('location', 'city', 'state', 'country'):
|
||||
place_name = geolocation.place_name(
|
||||
metadata['latitude'],
|
||||
metadata['longitude']
|
||||
)
|
||||
|
||||
location_parts = re.findall('(%[^%]+)', mask)
|
||||
parsed_folder_name = self.parse_mask_for_location(
|
||||
mask,
|
||||
location_parts,
|
||||
place_name,
|
||||
)
|
||||
path.append(parsed_folder_name)
|
||||
|
||||
# For now we always make the leaf folder an album if it's in the EXIF.
|
||||
# This is to preserve backwards compatability until we figure out how
|
||||
|
|
@ -217,11 +213,6 @@ class FileSystem(object):
|
|||
elif(len(path) == 2):
|
||||
path[1] = metadata['album']
|
||||
|
||||
# if we don't have a 2nd level directory we use 'Unknown Location'
|
||||
if(len(path) < 2):
|
||||
path.append('Unknown Location')
|
||||
|
||||
# return '/'.join(path[::-1])
|
||||
return os.path.join(*path)
|
||||
|
||||
def parse_mask_for_location(self, mask, location_parts, place_name):
|
||||
|
|
@ -338,11 +329,6 @@ class FileSystem(object):
|
|||
shutil.move(_file, dest_path)
|
||||
os.utime(dest_path, (stat.st_atime, stat.st_mtime))
|
||||
else:
|
||||
# Do not use copy2(), will have an issue when copying to a
|
||||
# network/mounted drive using copy and manual
|
||||
# set_date_from_filename gets the job done
|
||||
# shutil.copy seems slow, changing to streaming according to
|
||||
# http://stackoverflow.com/questions/22078621/python-how-to-copy-files-fast # noqa
|
||||
compatability._copyfile(_file, dest_path)
|
||||
self.set_utime(media)
|
||||
|
||||
|
|
@ -352,7 +338,7 @@ class FileSystem(object):
|
|||
return dest_path
|
||||
|
||||
def set_utime(self, media):
|
||||
""" Set the modification time on the file base on the file name.
|
||||
""" Set the modification time on the file based on the file name.
|
||||
"""
|
||||
|
||||
# Initialize date taken to what's returned from the metadata function.
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ from elodie import log
|
|||
from elodie.localstorage import Db
|
||||
|
||||
__KEY__ = None
|
||||
__DEFAULT_LOCATION__ = 'Unknown Location'
|
||||
|
||||
|
||||
def coordinates_by_name(name):
|
||||
|
|
@ -115,10 +116,14 @@ def get_key():
|
|||
|
||||
|
||||
def place_name(lat, lon):
|
||||
lookup_place_name_default = {'default': __DEFAULT_LOCATION__}
|
||||
if(lat is None or lon is None):
|
||||
return lookup_place_name_default
|
||||
|
||||
# Convert lat/lon to floats
|
||||
if not isinstance(lat, float):
|
||||
if(not isinstance(lat, float)):
|
||||
lat = float(lat)
|
||||
if not isinstance(lon, float):
|
||||
if(not isinstance(lon, float)):
|
||||
lon = float(lon)
|
||||
|
||||
# Try to get cached location first
|
||||
|
|
@ -132,19 +137,18 @@ def place_name(lat, lon):
|
|||
|
||||
lookup_place_name = {}
|
||||
geolocation_info = lookup(lat=lat, lon=lon)
|
||||
if(geolocation_info is not None):
|
||||
if('address' in geolocation_info):
|
||||
address = geolocation_info['address']
|
||||
for loc in ['city', 'state', 'country']:
|
||||
if(loc in address):
|
||||
lookup_place_name[loc] = address[loc]
|
||||
# In many cases the desired key is not available so we
|
||||
# set the most specific as the default.
|
||||
if('default' not in lookup_place_name):
|
||||
lookup_place_name['default'] = address[loc]
|
||||
if(geolocation_info is not None and 'address' in geolocation_info):
|
||||
address = geolocation_info['address']
|
||||
for loc in ['city', 'state', 'country']:
|
||||
if(loc in address):
|
||||
lookup_place_name[loc] = address[loc]
|
||||
# In many cases the desired key is not available so we
|
||||
# set the most specific as the default.
|
||||
if('default' not in lookup_place_name):
|
||||
lookup_place_name['default'] = address[loc]
|
||||
|
||||
if('default' not in lookup_place_name):
|
||||
lookup_place_name = {'default': 'Unknown Location'}
|
||||
lookup_place_name = lookup_place_name_default
|
||||
|
||||
if(lookup_place_name is not {}):
|
||||
db.add_location(lat, lon, lookup_place_name)
|
||||
|
|
|
|||
|
|
@ -248,6 +248,48 @@ full_path=%date/%location
|
|||
|
||||
assert path == os.path.join('2015-12-05','United States of America-California-Sunnyvale'), path
|
||||
|
||||
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
|
||||
def test_get_folder_path_with_with_more_than_two_levels():
|
||||
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
|
||||
f.write("""
|
||||
[Directory]
|
||||
year=%Y
|
||||
month=%m
|
||||
location=%city, %state
|
||||
full_path=%year/%month/%location
|
||||
""")
|
||||
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
filesystem = FileSystem()
|
||||
media = Photo(helper.get_file('with-location.jpg'))
|
||||
path = filesystem.get_folder_path(media.get_metadata())
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
assert path == os.path.join('2015','12','Sunnyvale, California'), path
|
||||
|
||||
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
|
||||
def test_get_folder_path_with_with_only_one_level():
|
||||
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
|
||||
f.write("""
|
||||
[Directory]
|
||||
year=%Y
|
||||
full_path=%year
|
||||
""")
|
||||
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
filesystem = FileSystem()
|
||||
media = Photo(helper.get_file('plain.jpg'))
|
||||
path = filesystem.get_folder_path(media.get_metadata())
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
assert path == os.path.join('2015'), path
|
||||
|
||||
def test_get_folder_path_with_location_and_title():
|
||||
filesystem = FileSystem()
|
||||
media = Photo(helper.get_file('with-location-and-title.jpg'))
|
||||
|
|
@ -658,3 +700,47 @@ full_path=%date/%location
|
|||
]
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
|
||||
def test_get_folder_path_definition_with_more_than_two_levels():
|
||||
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
|
||||
f.write("""
|
||||
[Directory]
|
||||
year=%Y
|
||||
month=%m
|
||||
day=%d
|
||||
full_path=%year/%month/%day
|
||||
""")
|
||||
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
filesystem = FileSystem()
|
||||
path_definition = filesystem.get_folder_path_definition()
|
||||
expected = [
|
||||
('year', '%Y'), ('month', '%m'), ('day', '%d')
|
||||
]
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
assert path_definition == expected, path_definition
|
||||
|
||||
@mock.patch('elodie.config.config_file', '%s/config.ini-location-date' % gettempdir())
|
||||
def test_get_folder_path_definition_with_only_one_level():
|
||||
with open('%s/config.ini-location-date' % gettempdir(), 'w') as f:
|
||||
f.write("""
|
||||
[Directory]
|
||||
year=%Y
|
||||
full_path=%year
|
||||
""")
|
||||
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
filesystem = FileSystem()
|
||||
path_definition = filesystem.get_folder_path_definition()
|
||||
expected = [
|
||||
('year', '%Y')
|
||||
]
|
||||
if hasattr(load_config, 'config'):
|
||||
del load_config.config
|
||||
|
||||
assert path_definition == expected, path_definition
|
||||
|
|
|
|||
Loading…
Reference in New Issue