- Remove shelve picture cache as it has had a fair number of historical
issues. Original issue for which it was added should be long
resolved. Additionally this allows additional consolidation of the
various cache code and potentially dbs in the future.
- Remove all related preferences and related code for changing cache
backend between sqlite and shelve.
- Add Callable type to hasher (should realy be more specific...)
- Add type hint to COLUMNS in qtlib/table.py
- Use Qt.ItemFlag.ItemIsEnabled instead of Qt.itemIsEnabled in qtlib/table.py
commit 8b15fe9a502ebf4841c6529e7098cef03a6a5e6f
Author: Andrew Senetar <arsenetar@gmail.com>
Date: Sun Mar 27 23:48:15 2022 -0500
Finish up changes to copy_or_move
commit 21f6a32cf3186a400af8f30e67ad2743dc9a49bd
Author: Andrew Senetar <arsenetar@gmail.com>
Date: Thu Mar 17 23:56:52 2022 -0500
Migrate from hscommon.path to pathlib
- Part one, this gets all hscommon and core tests passing
- App appears to be able to load directories and complete scans, need further testing
- app.py copy_or_move needs some additional work
commit 5eb515f666bfa1ff06c2e96bdc351a4b7456580e
Author: Andrew Senetar <arsenetar@gmail.com>
Date: Sun Mar 27 22:19:39 2022 -0500
Add fallback to md5 if xxhash not available
Mainly here for the case when distributions have not packaged python3-xxhash.
commit 51b18d4c84
Author: Andrew Senetar <arsenetar@gmail.com>
Date: Sat Mar 19 15:25:46 2022 -0500
Switch file hashing to xxhash instead of md5
- Improves performance significantly in some cases
- Add xxhash to requirements.txt and sort requirements
- Rename md5 based members to digest
- Update all tests to use new member names and hashing methods
- Update hash db code to upgrade schema
NOTE: May consider supporting multiple hashing algorithms in the future.
- Cleanup columns.py and tables
- Other misc cleanups
- Remove text_field.py from qtlib as it is not used
- Remove unused variables from image_viewer method
* Instead of keeping md5 samples separate, merge them as one hash computed from the various selected chunks we picked.
* We don't need to keep a boolean to see whether or not the user chose to optimize; we can simply compare the value of the threshold, since 0 means no optimization currently active.
* Doing a full match should be safer to avoid partial results which would result in overly aggressive filtering.
* Add new tests to test suite to cover this issue.
* Fixes#903.
* Accents are getting removed from Unicode characters to generate similar "words".
* Non-latin characters which cannot be processed that way (eg. japanese, greek, russian, etc.) should not be filtered out at all otherwise files are erroneously skipped or detected as dupes if only some characters make it passed the filter.
* Starting from an arbitrary unicode codepoint (converted to decimal), above which we know it is pointless to try any sort of processing, we leave the characters as is.
* Fix#878.
* Add testing feature to Exclusion dialog to allow users to test regexes against an arbitrary string.
* Fixed test suites.
* Improve comments and help dialog box.
- Currently some incompatibility in the hscommon tests, commented out
the ones with issues temporarily
- Also updated some deprecation warnings, still more to do
- Add changes from OSX build to local hscommon/build.py
- Update package.py & srcpkg.sh
- Remove invalid submodule references
- Update srcpkg.sh to use xz
- Update package.py pyinstaller configuration
- Call PyInstaller inline
- Add --noconfirm option to be more script friendly
- Add UCRT Redist location to path should fix#545 as now all the dlls
are included
- Format all files with black
- Update tox.ini flake8 arguments to be compatible
- Add black to requirements-extra.txt
- Reduce ignored flake8 rules and fix a few violations
These methods were previously called recursively and it seemed to cause
problems in some cases. The recursive nature of these functions not
bringing any notable advantage and `os.walk()` being of better style
anyway, I removed that recursive nature.
Hopefully fixes#421
We now get less progress feedback, but in exchange, our progress job is
simpler. Previously, our progress bar would often get wonky towards the
end of the scan and I didn't have the energy to debug that.
Besides, people don't care about that level of progress feedback.
(breaks PE temporarily)
Adds a Standard/Music Application Mode button to SE and thus adds the
ability to run ME scan types in SE. When in Music mode, the
Music-specific results window, details panel and preferences panel will
show up.
All preferences except scan_type become shared between app modes
(changing the pref in a mode changes it in the other mode).
Results Window and Details Panel are now re-created at each scan
operation because they could change their type between two runs.
Preferences panel is instantiated on the fly and discarded after close.
This is a very big merge operation and I'm trying to touch as little
code as possible, sometimes at the cost of elegance. I try to minimize
the breakage that this change brings.
It's now `DupeGuru` that holds it and passes it to `get_dupe_groups()`,
the only place where it's actually used in `Scanner`.
This will make the SE/ME/PE merge easier by allowing us to instantiate
the Scanner on-the-fly since it doesn't hold state anymore.
I have a fix to make in it and it's really silly to pretend that this
lib is of any use to anybody outside HS apps. Bringing it back here will
make things more simple.
