1
0
mirror of https://github.com/arsenetar/dupeguru.git synced 2024-10-30 05:15:57 +00:00
Commit Graph

37 Commits

Author SHA1 Message Date
e382683f66
Replace all relative imports 2022-05-09 01:40:08 -05:00
63dd4d4561
Apply pyupgrade changes 2022-04-27 20:53:12 -05:00
9f40e4e786
Squashed commit of the following:
commit 5eb515f666bfa1ff06c2e96bdc351a4b7456580e
Author: Andrew Senetar <arsenetar@gmail.com>
Date:   Sun Mar 27 22:19:39 2022 -0500

    Add fallback to md5 if xxhash not available

    Mainly here for the case when distributions have not packaged python3-xxhash.

commit 51b18d4c84
Author: Andrew Senetar <arsenetar@gmail.com>
Date:   Sat Mar 19 15:25:46 2022 -0500

    Switch file hashing to xxhash instead of md5

    - Improves performance significantly in some cases
    - Add xxhash to requirements.txt and sort requirements
    - Rename md5 based members to digest
    - Update all tests to use new member names and hashing methods
    - Update hash db code to upgrade schema

    NOTE: May consider supporting multiple hashing algorithms in the future.
2022-03-27 22:27:13 -05:00
e22d7d2fc9
Remove filtering of 0 size files in engine
Files size is already able to be filtered at a higher level, some users
may decide to see zero length files. Fix #321.
2021-08-28 18:16:22 -05:00
0189c29f47
Misc cleanups in core/tests 2021-08-21 03:52:09 -05:00
9446f37fad
Remove flake8 E731 Errors
Note: black formatting is now applying correctly as well.
2021-08-15 03:53:43 -05:00
glubsy
277bc3fbb8 Add unit tests for hash sample optimization
* Instead of keeping md5 samples separate, merge them as one hash computed from the various selected chunks we picked.
* We don't need to keep a boolean to see whether or not the user chose to optimize; we can simply compare the value of the threshold, since 0 means no optimization currently active.
2021-06-21 22:44:05 +02:00
glubsy
c4dcfd3d4b Fix stripping (japanese) unicode characters
* Accents are getting removed from Unicode characters to generate similar "words".
* Non-latin characters which cannot be processed that way (eg. japanese, greek, russian, etc.) should not be filtered out at all otherwise files are erroneously skipped or detected as dupes if only some characters make it passed the filter.
* Starting from an arbitrary unicode codepoint (converted to decimal), above which we know it is pointless to try any sort of processing, we leave the characters as is.
* Fix #878.
2021-04-29 05:15:34 +02:00
de8a0a21b2
Update Packaging
- Add changes from OSX build to local hscommon/build.py
- Update package.py & srcpkg.sh
  - Remove invalid submodule references
  - Update srcpkg.sh to use xz
- Update package.py pyinstaller configuration
  - Call PyInstaller inline
  - Add --noconfirm option to be more script friendly
  - Add UCRT Redist location to path should fix #545 as now all the dlls
    are included
2019-12-31 21:36:52 -06:00
7ba8aa3514
Format files with black
- Format all files with black
- Update tox.ini flake8 arguments to be compatible
- Add black to requirements-extra.txt
- Reduce ignored flake8 rules and fix a few violations
2019-12-31 20:16:27 -06:00
Virgil Dupras
9f2c3e7732 Fix failing test on OS X / py35
A 100 recursion limit was too low in that environment.
2016-07-01 15:29:50 -04:00
Virgil Dupras
b1ef3dc8fe Simplify progress report during scanning
We now get less progress feedback, but in exchange, our progress job is
simpler. Previously, our progress bar would often get wonky towards the
end of the scan and I didn't have the energy to debug that.

Besides, people don't care about that level of progress feedback.
2016-06-08 12:29:28 -04:00
Virgil Dupras
130581db53 Apply flake8 checks to tests 2016-05-29 15:02:39 -04:00
Virgil Dupras
e7076bc3bd Change license from BSD to GPLv3
See http://www.hardcoded.net/archive2014#2014-12-28 for context
2015-01-03 16:33:16 -05:00
Virgil Dupras
fc16ea8c49 Change copyright year to 2015 2015-01-03 16:30:57 -05:00
Virgil Dupras
ac32305532 Integrated the jobprogress library into hscommon
I have a fix to make in it and it's really silly to pretend that this
lib is of any use to anybody outside HS apps. Bringing it back here will
make things more simple.
2014-10-05 16:31:16 -04:00
Virgil Dupras
ca709a60cf Updated copyright year to 2014 2014-04-19 12:19:11 -04:00
Virgil Dupras
4a8ce9b6c4 Updated copyright year to 2013. 2013-04-28 10:35:51 -04:00
Virgil Dupras
4f097a3a89 [#204 state:fixed] Added a message after re-prioritization telling how many dupe groups were changed by it. 2012-07-31 11:37:51 -04:00
Virgil Dupras
657f6743c2 Changed copyright year to 2012 2012-03-15 14:28:40 -04:00
Virgil Dupras
9fac97c147 Don't allow dupes from ref folders to step down from their ref position during reprioritization. 2011-09-23 13:14:57 -04:00
Virgil Dupras
518228a368 [#138] Added a very preliminary version of the prioritization dialog. A big part of this commit is about refactoring the tests and introducing a TestApp for dupeGuru (in core.tests.base). 2011-09-07 15:46:41 -04:00
Virgil Dupras
0fea59007c Updated copyright year to 2011. 2011-04-12 10:04:01 +02:00
Virgil Dupras
eefe464fba Replaced dependencies from hsutil to hscommon. 2011-01-11 13:36:05 +01:00
Virgil Dupras
33c0ba808c Changed references to what has already been moved from hsutil to hscommon (io, path, testutil). 2011-01-11 11:59:53 +01:00
Virgil Dupras
e0cc8ecda2 Stop using hsutil.testcase. 2011-01-05 11:11:21 +01:00
Virgil Dupras
0382ad1534 Adapted to the job-related code moving to the 'jobprogress' package. 2010-11-20 12:42:15 +01:00
Virgil Dupras
4886982d43 Re-licensed to BSD 2010-09-30 12:17:41 +02:00
Virgil Dupras
854d194f88 Converted to py3k. There's probably some bugs still. So far, I managed to run dupeGuru SE under pyobjc and qt. 2010-08-11 16:39:06 +02:00
Virgil Dupras
4d66b4667c Moved from nose to py.test (the former doesn't officially support py3k, which is limiting). 2010-07-13 11:10:45 +02:00
Virgil Dupras
371426a08e Adapted codebase to the hsutil/hscommon split and the hsmedia --> hsaudiotag rename. 2010-07-13 08:08:18 +02:00
Virgil Dupras
75eb005ba0 Fixed a flaky test which was broken in python 2.7rc1. 2010-06-07 10:15:58 -04:00
Virgil Dupras
601b67145c Fixed a flaky test which was broken in python 2.7rc1. 2010-06-07 09:41:59 -04:00
Virgil Dupras
47a6ceffbc Use lxml everywhere for xml save/load (instead of ElementTree and minidom). 2010-03-01 12:21:43 +01:00
Virgil Dupras
d62ff40bed Removed svn keywords. 2010-01-02 16:52:18 +01:00
Virgil Dupras
c14fecb415 Changed copyright year to 2010 2010-01-01 21:11:34 +01:00
hsoft
838f8ae352 Changed the build system (that commit is *huge*)
--HG--
rename : base/cocoa/AppDelegate.h => cocoa/base/AppDelegate.h
rename : base/cocoa/AppDelegate.m => cocoa/base/AppDelegate.m
rename : base/cocoa/Consts.h => cocoa/base/Consts.h
rename : base/cocoa/DetailsPanel.h => cocoa/base/DetailsPanel.h
rename : base/cocoa/DetailsPanel.m => cocoa/base/DetailsPanel.m
rename : base/cocoa/DirectoryPanel.h => cocoa/base/DirectoryPanel.h
rename : base/cocoa/DirectoryPanel.m => cocoa/base/DirectoryPanel.m
rename : base/cocoa/PyDupeGuru.h => cocoa/base/PyDupeGuru.h
rename : base/cocoa/ResultWindow.h => cocoa/base/ResultWindow.h
rename : base/cocoa/ResultWindow.m => cocoa/base/ResultWindow.m
rename : base/cocoa/dsa_pub.pem => cocoa/base/dsa_pub.pem
rename : base/cocoa/xib/DetailsPanel.xib => cocoa/base/xib/DetailsPanel.xib
rename : base/cocoa/xib/DirectoryPanel.xib => cocoa/base/xib/DirectoryPanel.xib
rename : base/cocoa/xib/MainMenu.xib => cocoa/base/xib/MainMenu.xib
rename : me/cocoa/AppDelegate.h => cocoa/me/AppDelegate.h
rename : me/cocoa/AppDelegate.m => cocoa/me/AppDelegate.m
rename : me/cocoa/Consts.h => cocoa/me/Consts.h
rename : me/cocoa/DetailsPanel.h => cocoa/me/DetailsPanel.h
rename : me/cocoa/DetailsPanel.m => cocoa/me/DetailsPanel.m
rename : me/cocoa/DirectoryPanel.h => cocoa/me/DirectoryPanel.h
rename : me/cocoa/DirectoryPanel.m => cocoa/me/DirectoryPanel.m
rename : me/cocoa/Info.plist => cocoa/me/Info.plist
rename : me/cocoa/PyDupeGuru.h => cocoa/me/PyDupeGuru.h
rename : me/cocoa/ResultWindow.h => cocoa/me/ResultWindow.h
rename : me/cocoa/ResultWindow.m => cocoa/me/ResultWindow.m
rename : me/cocoa/dupeguru.icns => cocoa/me/dupeguru.icns
rename : me/cocoa/dupeguru.xcodeproj/project.pbxproj => cocoa/me/dupeguru.xcodeproj/project.pbxproj
rename : me/cocoa/gen.py => cocoa/me/gen.py
rename : me/cocoa/main.m => cocoa/me/main.m
rename : me/cocoa/py/dg_cocoa.py => cocoa/me/py/dg_cocoa.py
rename : me/cocoa/py/setup.py => cocoa/me/py/setup.py
rename : me/cocoa/xib/Preferences.xib => cocoa/me/xib/Preferences.xib
rename : pe/cocoa/AppDelegate.h => cocoa/pe/AppDelegate.h
rename : pe/cocoa/AppDelegate.m => cocoa/pe/AppDelegate.m
rename : pe/cocoa/Consts.h => cocoa/pe/Consts.h
rename : pe/cocoa/DetailsPanel.h => cocoa/pe/DetailsPanel.h
rename : pe/cocoa/DetailsPanel.m => cocoa/pe/DetailsPanel.m
rename : pe/cocoa/DirectoryPanel.h => cocoa/pe/DirectoryPanel.h
rename : pe/cocoa/DirectoryPanel.m => cocoa/pe/DirectoryPanel.m
rename : pe/cocoa/Info.plist => cocoa/pe/Info.plist
rename : pe/cocoa/PictureBlocks.h => cocoa/pe/PictureBlocks.h
rename : pe/cocoa/PictureBlocks.m => cocoa/pe/PictureBlocks.m
rename : pe/cocoa/PyDupeGuru.h => cocoa/pe/PyDupeGuru.h
rename : pe/cocoa/ResultWindow.h => cocoa/pe/ResultWindow.h
rename : pe/cocoa/ResultWindow.m => cocoa/pe/ResultWindow.m
rename : pe/cocoa/dupeguru.icns => cocoa/pe/dupeguru.icns
rename : pe/cocoa/dupeguru.xcodeproj/project.pbxproj => cocoa/pe/dupeguru.xcodeproj/project.pbxproj
rename : pe/cocoa/gen.py => cocoa/pe/gen.py
rename : pe/cocoa/main.m => cocoa/pe/main.m
rename : pe/cocoa/py/dg_cocoa.py => cocoa/pe/py/dg_cocoa.py
rename : pe/cocoa/py/setup.py => cocoa/pe/py/setup.py
rename : pe/cocoa/xib/DetailsPanel.xib => cocoa/pe/xib/DetailsPanel.xib
rename : pe/cocoa/xib/Preferences.xib => cocoa/pe/xib/Preferences.xib
rename : se/cocoa/AppDelegate.h => cocoa/se/AppDelegate.h
rename : se/cocoa/AppDelegate.m => cocoa/se/AppDelegate.m
rename : se/cocoa/Consts.h => cocoa/se/Consts.h
rename : se/cocoa/DetailsPanel.h => cocoa/se/DetailsPanel.h
rename : se/cocoa/DetailsPanel.m => cocoa/se/DetailsPanel.m
rename : se/cocoa/DirectoryPanel.h => cocoa/se/DirectoryPanel.h
rename : se/cocoa/DirectoryPanel.m => cocoa/se/DirectoryPanel.m
rename : se/cocoa/Info.plist => cocoa/se/Info.plist
rename : se/cocoa/PyDupeGuru.h => cocoa/se/PyDupeGuru.h
rename : se/cocoa/ResultWindow.h => cocoa/se/ResultWindow.h
rename : se/cocoa/ResultWindow.m => cocoa/se/ResultWindow.m
rename : se/cocoa/dupeguru.icns => cocoa/se/dupeguru.icns
rename : se/cocoa/dupeguru.xcodeproj/project.pbxproj => cocoa/se/dupeguru.xcodeproj/project.pbxproj
rename : se/cocoa/gen.py => cocoa/se/gen.py
rename : se/cocoa/main.m => cocoa/se/main.m
rename : se/cocoa/py/dg_cocoa.py => cocoa/se/py/dg_cocoa.py
rename : se/cocoa/py/setup.py => cocoa/se/py/setup.py
rename : se/cocoa/xib/Preferences.xib => cocoa/se/xib/Preferences.xib
rename : base/core/LICENSE => core/LICENSE
rename : base/core/__init__.py => core/__init__.py
rename : base/core/app.py => core/app.py
rename : base/core/app_cocoa.py => core/app_cocoa.py
rename : base/core/data.py => core/data.py
rename : base/core/directories.py => core/directories.py
rename : base/core/engine.py => core/engine.py
rename : base/core/export.py => core/export.py
rename : base/core/fs.py => core/fs.py
rename : base/core/ignore.py => core/ignore.py
rename : base/core/results.py => core/results.py
rename : base/core/scanner.py => core/scanner.py
rename : base/core/tests/__init__.py => core/tests/__init__.py
rename : base/core/tests/app_cocoa_test.py => core/tests/app_cocoa_test.py
rename : base/core/tests/app_test.py => core/tests/app_test.py
rename : base/core/tests/data.py => core/tests/data.py
rename : base/core/tests/directories_test.py => core/tests/directories_test.py
rename : base/core/tests/engine_test.py => core/tests/engine_test.py
rename : base/core/tests/ignore_test.py => core/tests/ignore_test.py
rename : base/core/tests/results_test.py => core/tests/results_test.py
rename : base/core/tests/scanner_test.py => core/tests/scanner_test.py
rename : me/core/__init__.py => core_me/__init__.py
rename : me/core/app_cocoa.py => core_me/app_cocoa.py
rename : me/core/data.py => core_me/data.py
rename : me/core/fs.py => core_me/fs.py
rename : me/core/scanner.py => core_me/scanner.py
rename : me/core/tests/__init__.py => core_me/tests/__init__.py
rename : me/core/tests/scanner_test.py => core_me/tests/scanner_test.py
rename : pe/core/LICENSE => core_pe/LICENSE
rename : pe/core/__init__.py => core_pe/__init__.py
rename : pe/core/app_cocoa.py => core_pe/app_cocoa.py
rename : pe/core/block.py => core_pe/block.py
rename : pe/core/cache.py => core_pe/cache.py
rename : pe/core/data.py => core_pe/data.py
rename : pe/core/gen.py => core_pe/gen.py
rename : pe/core/matchbase.py => core_pe/matchbase.py
rename : pe/core/modules/block/block.pyx => core_pe/modules/block/block.pyx
rename : pe/core/modules/block/setup.py => core_pe/modules/block/setup.py
rename : pe/core/modules/cache/cache.pyx => core_pe/modules/cache/cache.pyx
rename : pe/core/modules/cache/setup.py => core_pe/modules/cache/setup.py
rename : pe/core/scanner.py => core_pe/scanner.py
rename : pe/core/tests/__init__.py => core_pe/tests/__init__.py
rename : pe/core/tests/block_test.py => core_pe/tests/block_test.py
rename : pe/core/tests/cache_test.py => core_pe/tests/cache_test.py
rename : se/core/LICENSE => core_se/LICENSE
rename : se/core/__init__.py => core_se/__init__.py
rename : se/core/app_cocoa.py => core_se/app_cocoa.py
rename : se/core/data.py => core_se/data.py
rename : se/core/fs.py => core_se/fs.py
rename : se/core/tests/__init__.py => core_se/tests/__init__.py
rename : se/core/tests/fs_test.py => core_se/tests/fs_test.py
rename : me/help/LICENSE => help_me/LICENSE
rename : me/help/__init__.py => help_me/__init__.py
rename : me/help/changelog.yaml => help_me/changelog.yaml
rename : me/help/gen.py => help_me/gen.py
rename : me/help/skeleton/hardcoded.css => help_me/skeleton/hardcoded.css
rename : me/help/skeleton/images/hs_title.png => help_me/skeleton/images/hs_title.png
rename : me/help/templates/base_dg.mako => help_me/templates/base_dg.mako
rename : me/help/templates/credits.mako => help_me/templates/credits.mako
rename : me/help/templates/directories.mako => help_me/templates/directories.mako
rename : me/help/templates/faq.mako => help_me/templates/faq.mako
rename : me/help/templates/intro.mako => help_me/templates/intro.mako
rename : me/help/templates/power_marker.mako => help_me/templates/power_marker.mako
rename : me/help/templates/preferences.mako => help_me/templates/preferences.mako
rename : me/help/templates/quick_start.mako => help_me/templates/quick_start.mako
rename : me/help/templates/results.mako => help_me/templates/results.mako
rename : me/help/templates/versions.mako => help_me/templates/versions.mako
rename : pe/help/LICENSE => help_pe/LICENSE
rename : pe/help/__init__.py => help_pe/__init__.py
rename : pe/help/changelog.yaml => help_pe/changelog.yaml
rename : pe/help/gen.py => help_pe/gen.py
rename : pe/help/skeleton/hardcoded.css => help_pe/skeleton/hardcoded.css
rename : pe/help/skeleton/images/hs_title.png => help_pe/skeleton/images/hs_title.png
rename : pe/help/templates/base_dg.mako => help_pe/templates/base_dg.mako
rename : pe/help/templates/credits.mako => help_pe/templates/credits.mako
rename : pe/help/templates/directories.mako => help_pe/templates/directories.mako
rename : pe/help/templates/faq.mako => help_pe/templates/faq.mako
rename : pe/help/templates/intro.mako => help_pe/templates/intro.mako
rename : pe/help/templates/power_marker.mako => help_pe/templates/power_marker.mako
rename : pe/help/templates/preferences.mako => help_pe/templates/preferences.mako
rename : pe/help/templates/quick_start.mako => help_pe/templates/quick_start.mako
rename : pe/help/templates/results.mako => help_pe/templates/results.mako
rename : pe/help/templates/versions.mako => help_pe/templates/versions.mako
rename : se/help/LICENSE => help_se/LICENSE
rename : se/help/changelog.yaml => help_se/changelog.yaml
rename : se/help/gen.py => help_se/gen.py
rename : se/help/skeleton/hardcoded.css => help_se/skeleton/hardcoded.css
rename : se/help/skeleton/images/hs_title.png => help_se/skeleton/images/hs_title.png
rename : se/help/templates/base_dg.mako => help_se/templates/base_dg.mako
rename : se/help/templates/credits.mako => help_se/templates/credits.mako
rename : se/help/templates/directories.mako => help_se/templates/directories.mako
rename : se/help/templates/faq.mako => help_se/templates/faq.mako
rename : se/help/templates/intro.mako => help_se/templates/intro.mako
rename : se/help/templates/power_marker.mako => help_se/templates/power_marker.mako
rename : se/help/templates/preferences.mako => help_se/templates/preferences.mako
rename : se/help/templates/quick_start.mako => help_se/templates/quick_start.mako
rename : se/help/templates/results.mako => help_se/templates/results.mako
rename : se/help/templates/versions.mako => help_se/templates/versions.mako
rename : base/qt/WARNING => qt/WARNING
rename : base/qt/__init__.py => qt/base/__init__.py
rename : base/qt/app.py => qt/base/app.py
rename : base/qt/details_table.py => qt/base/details_table.py
rename : base/qt/dg.qrc => qt/base/dg.qrc
rename : base/qt/directories_dialog.py => qt/base/directories_dialog.py
rename : base/qt/directories_dialog.ui => qt/base/directories_dialog.ui
rename : base/qt/directories_model.py => qt/base/directories_model.py
rename : base/qt/main_window.py => qt/base/main_window.py
rename : base/qt/main_window.ui => qt/base/main_window.ui
rename : base/qt/platform.py => qt/base/platform.py
rename : base/qt/platform_osx.py => qt/base/platform_osx.py
rename : base/qt/platform_win.py => qt/base/platform_win.py
rename : base/qt/preferences.py => qt/base/preferences.py
rename : base/qt/results_model.py => qt/base/results_model.py
rename : me/qt/app.py => qt/me/app.py
rename : me/qt/build.py => qt/me/build.py
rename : me/qt/details_dialog.py => qt/me/details_dialog.py
rename : me/qt/details_dialog.ui => qt/me/details_dialog.ui
rename : me/qt/dgme.spec => qt/me/dgme.spec
rename : me/qt/gen.py => qt/me/gen.py
rename : me/qt/installer.aip => qt/me/installer.aip
rename : me/qt/preferences.py => qt/me/preferences.py
rename : me/qt/preferences_dialog.py => qt/me/preferences_dialog.py
rename : me/qt/preferences_dialog.ui => qt/me/preferences_dialog.ui
rename : me/qt/profile.py => qt/me/profile.py
rename : me/qt/start.py => qt/me/start.py
rename : me/qt/verinfo => qt/me/verinfo
rename : pe/qt/app.py => qt/pe/app.py
rename : pe/qt/block.py => qt/pe/block.py
rename : pe/qt/build.py => qt/pe/build.py
rename : pe/qt/details_dialog.py => qt/pe/details_dialog.py
rename : pe/qt/details_dialog.ui => qt/pe/details_dialog.ui
rename : pe/qt/dgpe.spec => qt/pe/dgpe.spec
rename : pe/qt/gen.py => qt/pe/gen.py
rename : pe/qt/installer.aip => qt/pe/installer.aip
rename : pe/qt/main_window.py => qt/pe/main_window.py
rename : pe/qt/modules/block/block.pyx => qt/pe/modules/block/block.pyx
rename : pe/qt/modules/block/setup.py => qt/pe/modules/block/setup.py
rename : pe/qt/preferences.py => qt/pe/preferences.py
rename : pe/qt/preferences_dialog.py => qt/pe/preferences_dialog.py
rename : pe/qt/preferences_dialog.ui => qt/pe/preferences_dialog.ui
rename : pe/qt/profile.py => qt/pe/profile.py
rename : pe/qt/start.py => qt/pe/start.py
rename : pe/qt/verinfo => qt/pe/verinfo
rename : se/qt/app.py => qt/se/app.py
rename : se/qt/build.py => qt/se/build.py
rename : se/qt/details_dialog.py => qt/se/details_dialog.py
rename : se/qt/details_dialog.ui => qt/se/details_dialog.ui
rename : se/qt/dgse.spec => qt/se/dgse.spec
rename : se/qt/gen.py => qt/se/gen.py
rename : se/qt/installer.aip => qt/se/installer.aip
rename : se/qt/preferences.py => qt/se/preferences.py
rename : se/qt/preferences_dialog.py => qt/se/preferences_dialog.py
rename : se/qt/preferences_dialog.ui => qt/se/preferences_dialog.ui
rename : se/qt/profile.py => qt/se/profile.py
rename : se/qt/start.py => qt/se/start.py
rename : se/qt/verinfo => qt/se/verinfo
extra : convert_revision : svn%3Ac306627e-7827-47d3-bdf0-9a457c9553a1/trunk%40285
2009-12-30 16:34:41 +00:00