1
0
mirror of https://github.com/arsenetar/dupeguru.git synced 2026-02-12 07:21:37 +00:00

Switch file hashing to xxhash instead of md5

- Improves performance significantly in some cases
- Add xxhash to requirements.txt and sort requirements
- Rename md5 based members to digest
- Update all tests to use new member names and hashing methods
- Update hash db code to upgrade schema

NOTE: May consider supporting multiple hashing algorithms in the future.
This commit is contained in:
2022-03-19 15:25:46 -05:00
parent bbcdfbf698
commit 51b18d4c84
9 changed files with 194 additions and 180 deletions

View File

@@ -238,7 +238,7 @@ def getmatches(pictures, cache_path, threshold, match_scaled=False, j=job.nulljo
for ref_id, other_id, percentage in myiter:
ref = id2picture[ref_id]
other = id2picture[other_id]
if percentage == 100 and ref.md5 != other.md5:
if percentage == 100 and ref.digest != other.digest:
percentage = 99
if percentage >= threshold:
ref.dimensions # pre-read dimensions for display in results