Commit Graph

344 Commits

Author SHA1 Message Date
Andrew Senetar 2c11eecf97
Update version and changelog to 4.2.0 2022-01-24 22:28:40 -06:00
Dobatymo 77460045c4 clean up abstraction 2021-10-29 15:24:47 +08:00
Dobatymo 9753afba74 change FilesDB to singleton class
move hash calculation back in to Files class
clear cache now clears hash cache in addition to picture cache
2021-10-29 15:12:40 +08:00
Dobatymo 1ea108fc2b changed cache filename 2021-10-29 15:12:40 +08:00
Dobatymo 2f02a6010d implement hash cache for md5 hash based on sqlite 2021-10-29 15:12:40 +08:00
Andrew Senetar 1d60e124ee
Update invoke_custom_command to run for all selected items 2021-09-02 20:48:25 -05:00
Andrew Senetar e22d7d2fc9
Remove filtering of 0 size files in engine
Files size is already able to be filtered at a higher level, some users
may decide to see zero length files. Fix #321.
2021-08-28 18:16:22 -05:00
Andrew Senetar 78fb052d77
Add more progress details to getmatches, ref #700 2021-08-28 04:58:22 -05:00
Andrew Senetar 9805cba10d
Use different message for direct delete success, close #904 2021-08-28 04:27:34 -05:00
Andrew Senetar 4c3dfe2f1f
Provide more feedback during scans
- Add output for number of collected files / folders
- Update to allow indeterminate progress bar
- Remove unused hscommon\jobprogress\qt.py
2021-08-28 04:05:07 -05:00
Andrew Senetar 3045361243
Add preference to ignore large files, close #430 2021-08-27 05:35:54 -05:00
Andrew Senetar 809116c764
Fix CodeQL Alerts
- Cast int to Py_ssize_t for multiplication
2021-08-26 03:43:31 -05:00
Andrew Senetar 47dbe805bb
More cleanup and fixed a flake8 build issue 2021-08-25 01:11:24 -05:00
Andrew Senetar f11fccc889
More cleanups
- Cleanup columns.py and tables
- Other misc cleanups
- Remove text_field.py from qtlib as it is not used
- Remove unused variables from image_viewer method
2021-08-25 00:46:33 -05:00
Andrew Senetar d576a7043c
Code cleanups in core and other affected files 2021-08-21 18:02:02 -05:00
Andrew Senetar 1ef5f56158
Code cleanups in hscommon & external effects 2021-08-21 16:56:27 -05:00
Andrew Senetar 0189c29f47
Misc cleanups in core/tests 2021-08-21 03:52:09 -05:00
Andrew Senetar 58c04ff9ad
Switch from hsaudiotag to mutagen, close #440
- This opens up the ability to support more tags and audio information
- Also makes progress on #333
2021-08-19 00:14:26 -05:00
Andrew Senetar be10b462fc
Add portable mode
If settings.ini is present next to the executable, will run in portable mode.
This results in settings, data, and cache all being in same folder as dupeGuru.
2021-08-17 21:12:32 -05:00
Andrew Senetar ffe6b7047c
Format all files with black correcting line length 2021-08-15 04:10:18 -05:00
Andrew Senetar 9446f37fad
Remove flake8 E731 Errors
Note: black formatting is now applying correctly as well.
2021-08-15 03:53:43 -05:00
Andrew Senetar e11f996dfc
Merge pull request #908 from glubsy/hash_sample_optimization
Hash sample optimization
2021-08-13 23:41:17 -05:00
glubsy e95306e58f Fix flake 8 2021-08-14 02:52:00 +02:00
glubsy 891a875990 Cache constant expression
Perhaps the python byte code is already optimized, but just in case it is not, keep pre-compute the constant expression.
2021-08-13 21:33:21 +02:00
glubsy 545a5a75fb Fix for older python versions
The "walrus" operator is only available in python 3.8 and later. Fall back to more traditional notation.
2021-08-13 20:56:33 +02:00
glubsy 7b764f183e Avoid partially hashing small files
Computing 3 hash samples for files less than 3MiB (3 * CHUNK_SIZE) is not efficient since spans of later samples would overlap a previous one.
Therefore we can simply return the hash of the entire small file instead.
2021-08-13 20:47:01 +02:00
glubsy 3dccb686e2 Fix Directories regex test
The entire path to the file would match unless another path separator is added.
2021-08-06 17:18:23 +02:00
Andrew Senetar 0db66baace
Merge pull request #907 from glubsy/missing_renamed_regex
Missing renamed regex
2021-08-03 22:26:08 -05:00
glubsy 23c59787e5 Fix infinite recursion
Force the Results to update its internal __dupes list whenever at least one group has re-prioritized and changed its dupes/ref.
2021-06-23 05:36:10 +02:00
glubsy a51f263632 Fix refs appearing in dupes-only view
* Some refs appeared in the dupes-only view after a re-prioritization was done a second time.
* It seems the core.Results.__dupes list was not properly updated whenever core.app.Dupeguru.reprioritize_groups() -> core.Results.sort_dupes() was called.
When a re-prioritization is done, some refs became dupe, and some dupes became ref in their place. So we need to update the new state of the internal list of dupes kept by the Results object, instead of relying on the outdated cached one.
* Fix #757.
2021-06-22 22:57:57 +02:00
glubsy 718ca5b313 Remove unused import 2021-06-22 02:41:33 +02:00
glubsy 277bc3fbb8 Add unit tests for hash sample optimization
* Instead of keeping md5 samples separate, merge them as one hash computed from the various selected chunks we picked.
* We don't need to keep a boolean to see whether or not the user chose to optimize; we can simply compare the value of the threshold, since 0 means no optimization currently active.
2021-06-21 22:44:05 +02:00
glubsy e07dfd5955 Add partial hashes optimization for big files
* Big files above the user selected threshold can be partially hashed in 3 places.
* If the user is willing to take the risk, we consider files with identical md5samples as being identical.
2021-06-21 19:03:21 +02:00
glubsy a6f83ad3d7 Fix missing regexp after rename
* Doing a full match should be safer to avoid partial results which would result in overly aggressive filtering.
* Add new tests to test suite to cover this issue.
* Fixes #903.
2021-06-19 02:00:25 +02:00
glubsy ab8750eedb Fix partial regex match yielding false positive 2021-06-17 03:49:59 +02:00
glubsy 22033211d6 Fix exception when deleting while in delta view 2021-05-31 23:49:21 +02:00
glubsy f1ae478433 Fix including character at the border 2021-04-29 05:29:35 +02:00
glubsy c4dcfd3d4b Fix stripping (japanese) unicode characters
* Accents are getting removed from Unicode characters to generate similar "words".
* Non-latin characters which cannot be processed that way (eg. japanese, greek, russian, etc.) should not be filtered out at all otherwise files are erroneously skipped or detected as dupes if only some characters make it passed the filter.
* Starting from an arbitrary unicode codepoint (converted to decimal), above which we know it is pointless to try any sort of processing, we leave the characters as is.
* Fix #878.
2021-04-29 05:15:34 +02:00
Andrew Senetar 4a40b346a4
Update to 4.1.1 2021-03-21 22:50:33 -05:00
glubsy 528dedd813 Fix problematic string for translations
Some languages have very different phrase syntaxes depending on which word is used.
Better used two separate strings than a dynamically created one.
2021-02-09 01:40:00 +01:00
Sergey Zhuravlevich 32dcd90b50 Prioritize dialog: allow removing multiple prioritizations at once
Removing prioritizations one-by-one can be tedious. This commit enables
extended selection in the prioritizations list. Multiple items can be
selected with conventional methods, such as holding down Ctrl or Shift
key and clicking the items or holding down the left mouse button and
hovering the cursor over the list. All items also can be selected with
Ctrl+A.

Multiple items drag-n-drop is also possible.

To avoid confusion, the selection in the prioritizations list is cleared
after the items are removed or drag-n-dropped.

Signed-off-by: Sergey Zhuravlevich <sergey@zhur.xyz>
2021-01-07 17:42:30 +01:00
Sergey Zhuravlevich c2fef8d624 Prioritize dialog: allow adding multiple criteria at once
Adding criteria to the prioritizations list one-by-one can be tedious.
This commit enables extended selection in the criteria list and
implements adding multiple items. Multiple criteria can be selected with
conventional methods, such as holding down Ctrl or Shift keys and
clicking the items or holding down the left mouse button and hovering
the cursor over the list. All items also can be selected with Ctrl+A.

Signed-off-by: Sergey Zhuravlevich <sergey@zhur.xyz>
2021-01-07 17:42:07 +01:00
glubsy b138dfad33 Fix exception when testing invalid regex
* If a regex in the table is invalid and failed to compile, its "compiled" property is None.
* Only test against the regex if its compilation worked.
2020-12-30 22:50:42 +01:00
glubsy c1d94d6771 Merge branch 'master' into dev 2020-12-29 20:10:42 +01:00
glubsy f0d3dec517 Fix exclude tests 2020-12-29 16:07:55 +01:00
glubsy e533a396fb Remove redundant check 2020-12-29 05:39:26 +01:00
glubsy 4b4cc04e87 Fix directories tests on Windows
Regexes did not match properly because the separator for Windows is '\\'
2020-12-29 05:35:30 +01:00
glubsy 6bc619055e Change version to 4.1.0 2020-12-06 20:13:03 +01:00
glubsy 680cb581c1 Merge branch 'master' into exclude_list 2020-10-28 03:58:05 +01:00
glubsy 32d66cd19b Move up to 4.0.5
* Initial push to 4.0.5 milestone
* Update changelog
2020-10-27 19:38:51 +01:00