385 Commits

Author	SHA1	Message	Date
Andrew Senetar	99ec4e0f27	fix: Minor cleanups and fixes ... - Update NullJob to subclass Job - Remove unnecessary size pre-read in _getMatches() as file sizes are already loaded during file scan via stat call - Skip ref check if contents scan as the scan already prevents this from happening, some of the other scans do things differently and need to be reviewed before removing this post step completely - Add guard on partial hashing to just hash the whole file if smaller than the offset and size and use the value for both the partial digest and digest	2023-06-08 01:14:52 -05:00
Andrew Senetar	6d8b86b7eb	fix(core): Remove old directory state logic ... - Remove code forcing the exclusion of `.` directories by default, the new default exclusion filters do this by default - Change default state code to always return a value	2023-02-27 17:58:15 -06:00
Andrew Senetar	46521c8af1	feat: Add migration for picture cache db ... - Add migration (just delete db and change to new schema) for picture cache following the same sort of strategy as the file digest cache - Rename mtime column to mtime_ns to match file cache for consistency	2023-01-13 00:05:47 -06:00
Andrew Senetar	8c5e18b980	Merge remote-tracking branch 'upstream/master' into colors-bytes	2023-01-12 00:14:17 -06:00
Andrew Senetar	c57042fdd2	fix: Resolve issue with mock object for core test ... Last change introduced a new method on the fs.File object that the test object did not have. Add similar method to test object.	2023-01-11 23:20:40 -06:00
Andrew Senetar	057be0294a	fix: Prevent exception during existence check ... - Add "safe" existence check to files which catches OSErrors that may occur when trying to stat files - Use "safe" existence check during final existence check	2023-01-11 23:07:06 -06:00
Andrew Senetar	81daddd072	refactor: Improve digest cache db method performance ... - Remove lock on read operations, only needed for write operations - Change to use context manager for sqlite connection - Remove long lived cursor object and use short lived cursors instead Fixes #1080	2023-01-11 00:58:29 -06:00
Andrew Senetar	1e651a1603	Merge pull request #1089 from arsenetar/as/pre-commit ... feat: Add pre-commit, include python 3.11 in tests	2023-01-09 23:18:13 -06:00
Andrew Senetar	46d1afb566	chore: Apply whitespace fixes from hooks ... - Remove trailing whitespace - Correct single newline at end of files (skip for json) - Update to formatting in a few places due to black	2023-01-09 22:58:08 -06:00
Andrew Senetar	6db2fa2be6	fix: Correct flake8 config ... - Add exclude pattern for flake8 when running with pre-commit as it does not fully honor the exclude paths. - Cleanup exclude paths for flake8 in tox.ini - Re-enable line length check and correct three affected files	2023-01-09 22:35:12 -06:00
Andrew Senetar	83f5e80427	feat: Remove shelve picture cache ... - Remove shelve picture cache as it has had a fair number of historical issues. Original issue for which it was added should be long resolved. Additionally this allows additional consolidation of the various cache code and potentially dbs in the future. - Remove all related preferences and related code for changing cache backend between sqlite and shelve.	2023-01-06 00:35:23 -06:00
Andrew Senetar	e30a135451	feat: Add additional scan time options ... - Add option to include file existence check at end of scan, speeds up end of scan operation time considerably, however if user has removed or moved files since starting a scan there could be later errors when interacting with results. Defaults to existing behavior of including the check, until it can be verified later dialogs and actions handle non-existent items better. - Add option to ignore differences in mtime when checking hash cache. Option is present in advanced tab of preferences. Closes #1022. - Regenerate pot files for translations	2023-01-05 23:01:16 -06:00
Eugene San (eugenesan)	c920412856	Add webp image format support	2022-11-24 13:53:27 -07:00
Dobatymo	f1153c85c0	serialize/deserialize colors to/from bytes instead of strings ... it's a tiny bit faster and saves a bit of memory	2022-09-27 17:47:38 +08:00
Andrew Senetar	1f1dfa88dc	Update version & changelog for 4.3.1 release	2022-07-07 22:06:06 -05:00
Andrew Senetar	71af825b37	Move try/except of cache db to get() and put() ... - Move the try/except of cache db calls to the calls themselves. - Add some additional information to logging statements on cache db exception to improve troubleshooting.	2022-07-07 21:52:22 -05:00
Andrew Senetar	97f490b8b7	Fix typo in engine.py	2022-07-07 19:06:35 -05:00
Andrew Senetar	d369bcddd7	Updates from investigation of #1015 ... - Add protection for empty hash digests in comparison of non-zero size files - Bump version to 4.3.1-dev for identification	2022-07-07 19:00:09 -05:00
Andrew Senetar	360dceca7b	Update to version 4.3.0, update changelog	2022-06-30 23:27:14 -05:00
Andrew Senetar	92b27801c3	Update translations, remove iphoto_plist.py	2022-06-30 23:03:40 -05:00
Marcus Yanello	b9aabb8545	Redirect stdout from custom command to the log files (#1008) ... Send the logs for the custom command subprocess to the logs Closes #1007	2022-06-13 21:04:40 -05:00
Andrew Senetar	58863b1728	Change to use a real temporary directory for test ... app_test was not using a real temporary location originally	2022-05-09 01:46:42 -05:00
Andrew Senetar	e382683f66	Replace all relative imports	2022-05-09 01:40:08 -05:00
Andrew Senetar	f7ed1c801c	Add type hinting to desktop.py	2022-05-09 01:15:25 -05:00
Andrew Senetar	0a4e61edf5	Additional cleanup per mypy ... - Add Callable type to hasher (should realy be more specific...) - Add type hint to COLUMNS in qtlib/table.py - Use Qt.ItemFlag.ItemIsEnabled instead of Qt.itemIsEnabled in qtlib/table.py	2022-04-30 05:16:46 -05:00
Andrew Senetar	d73a85b82e	Add type hints for compiled modules	2022-04-30 05:11:54 -05:00
Andrew Senetar	63dd4d4561	Apply pyupgrade changes	2022-04-27 20:53:12 -05:00
Andrew Senetar	c5818b1d1f	Add option to profile scans ... - Add preference for profiling scans - Move debug options to tab in preferences - Add label with clickable link to debug output (appdata) to debug tab in preferences - Update translation source files	2022-03-31 00:16:37 -05:00
Andrew Senetar	a470a8de25	Update fs.py to optimize stat() calls ... - Update to get size and mtime at time of class creation when os.DirEntry is used for initialization. - Folders still calculate size later for folder scans. - Ref #962, #959	2022-03-30 22:58:01 -05:00
Andrew Senetar	a37b5b0eeb	Fix #988	2022-03-30 01:06:51 -05:00
Andrew Senetar	efd500ecc1	Update directory scanning to use os.scandir() ... - Change to use os.scandir() instead of os.walk() to leverage DirEntry objects. - Avoids extra calls to stat() on files during fs.can_handle() - See 3x speed improvement on Windows in some cases	2022-03-29 23:37:56 -05:00
Andrew Senetar	43fcc52291	Replace pathlib.glob() with os.scandir() in fs.py	2022-03-29 22:35:38 -05:00
Andrew Senetar	50f5db1543	Update fs to support DirEntry on get_file()	2022-03-29 22:32:36 -05:00
Andrew Senetar	a5b0ccdd02	Improve performance of Directories.get_state()	2022-03-29 21:48:14 -05:00
Andrew Senetar	ebb81d9f03	Remove pathlib function added in Python 3.9	2022-03-28 00:06:32 -05:00
Andrew Senetar	da9f8b2b9d	Squashed commit of the following: ... commit 8b15fe9a502ebf4841c6529e7098cef03a6a5e6f Author: Andrew Senetar <arsenetar@gmail.com> Date: Sun Mar 27 23:48:15 2022 -0500 Finish up changes to copy_or_move commit 21f6a32cf3186a400af8f30e67ad2743dc9a49bd Author: Andrew Senetar <arsenetar@gmail.com> Date: Thu Mar 17 23:56:52 2022 -0500 Migrate from hscommon.path to pathlib - Part one, this gets all hscommon and core tests passing - App appears to be able to load directories and complete scans, need further testing - app.py copy_or_move needs some additional work	2022-03-27 23:50:03 -05:00
Andrew Senetar	9f40e4e786	Squashed commit of the following: ... commit 5eb515f666bfa1ff06c2e96bdc351a4b7456580e Author: Andrew Senetar <arsenetar@gmail.com> Date: Sun Mar 27 22:19:39 2022 -0500 Add fallback to md5 if xxhash not available Mainly here for the case when distributions have not packaged python3-xxhash. commit 51b18d4c84 Author: Andrew Senetar <arsenetar@gmail.com> Date: Sat Mar 19 15:25:46 2022 -0500 Switch file hashing to xxhash instead of md5 - Improves performance significantly in some cases - Add xxhash to requirements.txt and sort requirements - Rename md5 based members to digest - Update all tests to use new member names and hashing methods - Update hash db code to upgrade schema NOTE: May consider supporting multiple hashing algorithms in the future.	2022-03-27 22:27:13 -05:00
Andrew Senetar	86bf9b39d0	Add update check function and call from about ... - Implement a update check against the GitHub releases via the api - Add semantic-version dependency - Add automatic check when opening about dialog	2022-03-27 21:13:27 -05:00
Andrew Senetar	1bc206e62d	Bump version to 4.2.1	2022-03-19 19:02:41 -05:00
Andrew Senetar	cbfa8720f1	Update imports for objc module	2022-03-09 05:01:12 -06:00
Andrew Senetar	85e22089bd	Black formatting changes	2022-02-09 21:49:51 -06:00
Andrew Senetar	2c11eecf97	Update version and changelog to 4.2.0	2022-01-24 22:28:40 -06:00
Dobatymo	77460045c4	clean up abstraction	2021-10-29 15:24:47 +08:00
Dobatymo	9753afba74	change FilesDB to singleton class ... move hash calculation back in to Files class clear cache now clears hash cache in addition to picture cache	2021-10-29 15:12:40 +08:00
Dobatymo	1ea108fc2b	changed cache filename	2021-10-29 15:12:40 +08:00
Dobatymo	2f02a6010d	implement hash cache for md5 hash based on sqlite	2021-10-29 15:12:40 +08:00
Andrew Senetar	1d60e124ee	Update invoke_custom_command to run for all selected items	2021-09-02 20:48:25 -05:00
Andrew Senetar	e22d7d2fc9	Remove filtering of 0 size files in engine ... Files size is already able to be filtered at a higher level, some users may decide to see zero length files. Fix #321.	2021-08-28 18:16:22 -05:00
Andrew Senetar	78fb052d77	Add more progress details to getmatches, ref #700	2021-08-28 04:58:22 -05:00
Andrew Senetar	9805cba10d	Use different message for direct delete success, close #904	2021-08-28 04:27:34 -05:00