1
0
mirror of https://github.com/arsenetar/dupeguru.git synced 2024-10-31 22:05:58 +00:00
Commit Graph

347 Commits

Author SHA1 Message Date
glubsy
6bc619055e Change version to 4.1.0 2020-12-06 20:13:03 +01:00
glubsy
680cb581c1 Merge branch 'master' into exclude_list 2020-10-28 03:58:05 +01:00
glubsy
32d66cd19b Move up to 4.0.5
* Initial push to 4.0.5 milestone
* Update changelog
2020-10-27 19:38:51 +01:00
glubsy
2875448c71 Merge branch 'save_directories' into dev 2020-10-27 16:23:49 +01:00
glubsy
424d34a7ed Add desktop.ini to filter list 2020-09-04 19:07:07 +02:00
glubsy
2a032d24bc Save/Load directories in Directories
* Add the ability to save / load directories as XML, just like the last_directories.xml which get loaded on program start.
2020-09-04 18:56:25 +02:00
glubsy
ea11a566af Highlight rows when testing regex string
* Add testing feature to Exclusion dialog to allow users to test regexes against an arbitrary string.
* Fixed test suites.
* Improve comments and help dialog box.
2020-09-01 23:02:58 +02:00
glubsy
4a1641e39d Add test suite, fix bugs 2020-08-31 20:35:56 +02:00
glubsy
9f223f3964 Concatenate regexes prio to compilation
* Concatenating regexes into one Pattern might yield better performance under (un)certain conditions.
* Filenames are tested against regexes with no os.sep in them. This may or may not be what we want to do.
And alternative would be to test against the whole (absolute) path of each file, which would filter more agressively.
2020-08-20 02:46:06 +02:00
glubsy
2eaf7e7893 Implement exclude list dialog on the Qt side 2020-08-17 05:54:59 +02:00
glubsy
a26de27c47 Implement dialog and base classes for model/view 2020-08-14 20:19:47 +02:00
glubsy
470307aa3c Ignore path and filename based on regex
* Added initial draft for test suit
* Fixed small logging bug
2020-08-03 16:19:27 +02:00
glubsy
5f5f9232c1 Properly wait for multiprocesses to exit
* Fix for #693
2020-07-28 16:44:06 +02:00
glubsy
63b2f95cfa Work around frozen progress dialog
* It seems that matchblock.getmatches() returns too early and the (multi-)processes become zombies
* This is a workaround which seems to work by sleeping for one second and avoid zombie processes
2020-07-25 23:37:41 +02:00
5cc439d846
Clean up rest of DeprecationWarnings 2020-06-30 00:51:06 -05:00
ee2671a5f3
More Test and Flake8 Cleanup
- Allow flake8 to check more files as well.
2020-06-27 01:08:12 -05:00
e05c72ad8c
Upgrade to latest pytest
- Currently some incompatibility in the hscommon tests, commented out
the ones with issues temporarily
- Also updated some deprecation warnings, still more to do
2020-06-25 23:26:48 -05:00
glubsy
bcb26507fe Remove superfluous argument 2020-06-25 01:23:03 +02:00
glubsy
ed64428c80 Add missing file class for folder type.
* results.py doesn't set the proper type for dupes at the line
"file = get_file(path)" so we add it on top
* Perhap it could have been added to _get_fileclasses() in core.app.py too
but I have not tested it
2020-06-24 23:32:04 +02:00
glubsy
e89156e55c Add temporary workaround for bug #676
* In standard mode, for folder comparison, dupe type is wrongly set as core.fs.Folder
while it should be core.se.fs.Folder.
* Catching the NotImplementedError exception redirects to the appropriate handler
* This is only a temporary workaround until a better fix is implemented
2020-06-24 22:01:30 +02:00
de8a0a21b2
Update Packaging
- Add changes from OSX build to local hscommon/build.py
- Update package.py & srcpkg.sh
  - Remove invalid submodule references
  - Update srcpkg.sh to use xz
- Update package.py pyinstaller configuration
  - Call PyInstaller inline
  - Add --noconfirm option to be more script friendly
  - Add UCRT Redist location to path should fix #545 as now all the dlls
    are included
2019-12-31 21:36:52 -06:00
7ba8aa3514
Format files with black
- Format all files with black
- Update tox.ini flake8 arguments to be compatible
- Add black to requirements-extra.txt
- Reduce ignored flake8 rules and fix a few violations
2019-12-31 20:16:27 -06:00
Eugene San (eugenesan)
080bb8935c Update packaging for 4.0.4
* Fix main version (Don't use spaces and capitals in versions!)
* Change debian changelog format in hscommon
* Fix build cleanup
* Switch to XZ compression
* Update build instructions
* Build single package for both Debian/Ubuntu
* Update packaging
2019-08-29 14:50:41 -07:00
c661905350
Bump version to 4.0.4 RC
- Also update package.py to allow version postfixes without causing
  issues with Windows build.
2019-05-13 20:18:56 -05:00
1a04f6ee86
Update to fix Travis Build & flake8 warning
- Fix Travis build by using dist: xenial so python 3.7 is available
- Correct flake8 warning in core/test/app_test.py
2018-12-04 20:16:15 -06:00
Virgil Dupras
d5fef949e9 directories: un-recurse get_files() and get_state()
These methods were previously called recursively and it seemed to cause
problems in some cases. The recursive nature of these functions not
bringing any notable advantage and `os.walk()` being of better style
anyway, I removed that recursive nature.

Hopefully fixes #421
2017-10-08 20:35:58 -04:00
Virgil Dupras
2f31dc7aab cache_shelve: wrap deletions in try..except in purge_outdated
Hopefully solves #402 and #439.
2017-09-19 13:22:33 -04:00
Virgil Dupras
6a28017c49 v4.0.3 2016-11-25 01:04:31 +00:00
Virgil Dupras
dc6933c90c Fix crash when cleaning picture cache 2016-11-25 00:59:51 +00:00
Virgil Dupras
76cc2000ab Add UI preference to picture cache type under Qt 2016-11-22 02:41:43 +00:00
Virgil Dupras
c58a4817ca Add shelve-based picture cache implementation
Hopefully, this will fix #394 for real this time, that is, without the
need for a messy python executable ship in the app.
2016-11-15 19:58:18 -05:00
Virgil Dupras
cc01e8eb09 Move pe.cache.Cache into its own unit, cache_sqlite
This prepares us for an upcoming alternative cache implementation.
2016-11-13 17:01:20 -05:00
Virgil Dupras
1c20e5c770 v4.0.2 2016-10-09 12:32:04 -04:00
Virgil Dupras
6ed4499a97 v4.0.1 2016-08-24 20:31:58 -04:00
Virgil Dupras
ac8a336c4a Fix picture mode's fuzzy block scanner threshold
It was always wrongly set to it's weirld old default 75 threshold.

fixes #387
2016-08-22 21:35:46 -04:00
Virgil Dupras
9f2c3e7732 Fix failing test on OS X / py35
A 100 recursion limit was too low in that environment.
2016-07-01 15:29:50 -04:00
Virgil Dupras
df9f72d9bf v4.0.0 2016-06-10 09:16:54 -04:00
Virgil Dupras
b1ef3dc8fe Simplify progress report during scanning
We now get less progress feedback, but in exchange, our progress job is
simpler. Previously, our progress bar would often get wonky towards the
end of the scan and I didn't have the energy to debug that.

Besides, people don't care about that level of progress feedback.
2016-06-08 12:29:28 -04:00
Virgil Dupras
334f4dd2ae Increase md5 reading buffer to 1mb
This makes md5 computing faster without using too much memory.
2016-06-08 12:23:10 -04:00
Virgil Dupras
fbdd1d866e Simplify getmatches_by_contents() signature
partial and sizeattr attributes are not needed anymore.
2016-06-08 12:06:08 -04:00
Virgil Dupras
80f659858c Fail with excplicit message when unable to load results file
Previously, we would simply show an empty results window. Not very
helpful.
2016-06-07 21:34:04 -04:00
Virgil Dupras
ef8f8f0e44 Fix broken tests 2016-06-07 21:32:30 -04:00
Virgil Dupras
b7a7282c2a Fix results loading
The merge operation broke it. It would try to access a result_table that
didn't exist yet.
2016-06-07 16:56:59 -04:00
Virgil Dupras
13fb06a693 Remove ContentsAusio scan type
It had few uses and had a confusing name. People though it did fuzzy
audio data matching, which it does not.
2016-06-06 17:08:41 -04:00
Virgil Dupras
8c1078aa71 cocoa: merge se/me/pe into one single app
That merge has already been done in core and qt, we're following.

I broke picture scan details panel image loading. Will fix later.
2016-06-05 21:18:48 -04:00
Virgil Dupras
a65077f871 Merge core_{se,me,pe} into core.{se,me,pe} 2016-05-31 22:32:37 -04:00
Virgil Dupras
d4919054f9 qt: move qt.base units into qt root package 2016-05-31 21:59:31 -04:00
Virgil Dupras
773f6651e6 Merge core_se.app into core.app 2016-05-31 21:43:24 -04:00
Virgil Dupras
8c9ef3ea29 Re-add the Clear Picture Cache action 2016-05-31 20:55:32 -04:00
Virgil Dupras
7d749779f2 qt: merge ME edition into SE
(breaks PE temporarily)

Adds a Standard/Music Application Mode button to SE and thus adds the
ability to run ME scan types in SE. When in Music mode, the
Music-specific results window, details panel and preferences panel will
show up.

All preferences except scan_type become shared between app modes
(changing the pref in a mode changes it in the other mode).

Results Window and Details Panel are now re-created at each scan
operation because they could change their type between two runs.

Preferences panel is instantiated on the fly and discarded after close.

This is a very big merge operation and I'm trying to touch as little
code as possible, sometimes at the cost of elegance. I try to minimize
the breakage that this change brings.
2016-05-29 22:37:38 -04:00
Virgil Dupras
0056f696df refactoring: move fileclasses and folderclass options in app class
Previously, it was in `Directory`.

This will make our job easier for an upcoming SE/ME/PE merge.
2016-05-29 17:15:55 -04:00
Virgil Dupras
5c57a2a8fc Instantiate Scanner on-the-fly
Previously, it would be instantiated on startup.

This will make our job easier for an upcoming SE/ME/PE merge.
2016-05-29 16:52:07 -04:00
Virgil Dupras
130581db53 Apply flake8 checks to tests 2016-05-29 15:02:39 -04:00
Virgil Dupras
9ed4b7abf0 refactoring: take ignore_list out of Scanner class
It's now `DupeGuru` that holds it and passes it to `get_dupe_groups()`,
the only place where it's actually used in `Scanner`.

This will make the SE/ME/PE merge easier by allowing us to instantiate
the Scanner on-the-fly since it doesn't hold state anymore.
2016-05-29 14:13:19 -04:00
Virgil Dupras
2ed1b82ecf Push edition-specific scan option listing down to the core
... rather than have each UI layer repeat them.

Did qt, but not cocoa yet.
2016-05-24 22:53:03 -04:00
Virgil Dupras
e7076bc3bd Change license from BSD to GPLv3
See http://www.hardcoded.net/archive2014#2014-12-28 for context
2015-01-03 16:33:16 -05:00
Virgil Dupras
fc16ea8c49 Change copyright year to 2015 2015-01-03 16:30:57 -05:00
Virgil Dupras
bf09c4ce8a Nicely wrap PermissionDenied errors on save
In fact, all `OSError`.

ref #266
2014-10-17 15:46:43 -04:00
Virgil Dupras
2166a0996c Added tox configuration
... and fixed pep8 warnings. There's a lot of them that are still
ignored, but that's because it's too much of a step to take at once.
2014-10-13 15:08:59 -04:00
Virgil Dupras
ac32305532 Integrated the jobprogress library into hscommon
I have a fix to make in it and it's really silly to pretend that this
lib is of any use to anybody outside HS apps. Bringing it back here will
make things more simple.
2014-10-05 16:31:16 -04:00
Virgil Dupras
878205fc49 Fix empty ignore List dialog bug in PE
Re-instantiating a new scanner for PE  made the ignore list dialog
target the wrong ignore list. We now only instantiate a scanner once.

Fixes #253
2014-05-03 13:44:38 -04:00
Virgil Dupras
ca709a60cf Updated copyright year to 2014 2014-04-19 12:19:11 -04:00
Virgil Dupras
781f13ae1a Overwrite subfolders' state when setting states in folder dialog
Fixes #248
2014-03-15 17:31:33 -04:00
Virgil Dupras
418acf6e5e Merge branch 'regless' into develop
Conflicts:
	cocoa/inter/app.py
	core/app.py
	hscommon/reg.py
	locale/cs/LC_MESSAGES/ui.po
	locale/de/LC_MESSAGES/ui.po
	locale/fr/LC_MESSAGES/ui.po
	locale/hy/LC_MESSAGES/ui.po
	locale/it/LC_MESSAGES/ui.po
	locale/pt_BR/LC_MESSAGES/ui.po
	locale/ru/LC_MESSAGES/ui.po
	locale/ui.pot
	locale/uk/LC_MESSAGES/ui.po
	locale/vi/LC_MESSAGES/ui.po
	locale/zh_CN/LC_MESSAGES/ui.po
	qt/base/app.py
2013-12-07 10:19:31 -05:00
Virgil Dupras
cb8bb5a70e Disable symlink/hardlink option when not relevant (Qt)
When the "Replace with links" option is not enabled, the choice of
symlink or hardlink is irrelevant and causes confusion. Implemented core
mechanism for controlling the enabled state of that option. Also
implemented the Qt interface for it. Cocoa-part is still to be done.

I used this opportunity to greatly enhance documentation of this part of
the code. I'm beginning to like documenting...

Ref #247.
2013-12-06 15:48:01 -05:00
Virgil Dupras
01403a3f92 Removed fairware 2013-11-30 17:54:40 -05:00
Virgil Dupras
c1cfa86ad1 Make Cmd+A select all folders in the Folder Selection dialog (Cocoa)
Fixes #228.
2013-11-24 10:12:47 -05:00
Virgil Dupras
c34c9562d3 Make non-numeric delta comparison case insensitive
Fixes #239.
2013-11-23 15:31:20 -05:00
Virgil Dupras
0e542577b0 Merge branch 'master' into develop 2013-11-23 12:39:59 -05:00
Virgil Dupras
42be49da83 Fix surrogate-related UnicodeEncodeError on CSV export
Fixes #210.
2013-11-23 12:38:55 -05:00
Virgil Dupras
10dbfa9b38 Refactoring: Path API compatibility with pathlib
Refactored dupeGuru to make hscommon.path's API a bit close to pathlib's
API. It's not 100% compatible yet, but it's much better than before.

This is more of a hscommon refactoring than a dupeguru one, but since
duepGuru is the main user of Path, it was the driver behind the
refactoring.

This refactoring also see the introduction of @pathify, which ensure
Path arguments. Previously, we were often unsure of whether the caller
of a function was passing a Path or a str. This problem is now solved
and this allows us to remove hscommon.io, an ill-conceived attempt to
solve that same ambiguity problem.

Fixes #235.
2013-11-16 12:06:16 -05:00
Virgil Dupras
e8c42740cf Fixed tests which were broken 2013-11-10 12:54:35 -05:00
Virgil Dupras
76f45fb5a6 Fixed appdata logic which was broken on OS X. 2013-11-10 11:05:03 -05:00
Virgil Dupras
12cf9b800b Merge branch 'master' into develop 2013-11-09 16:21:59 -05:00
Virgil Dupras
ba7e6494c6 Fixed crash on Dupe Count sorting with Delta + Dupes Only
Fixes #238
2013-11-09 16:20:33 -05:00
Virgil Dupras
72d8160b28 Fix boken tests 2013-11-08 16:45:14 -05:00
Virgil Dupras
6d53511cee Merge branch 'master' into develop 2013-11-08 16:03:35 -05:00
Virgil Dupras
8e65f15e1a Fixed core.engine.Match docstring
The way it was set made dupeGuru crash under Python 3.2
2013-10-20 13:33:27 -04:00
Virgil Dupras
8efefaf0bf Improved API docs 2013-10-12 13:55:36 -04:00
Virgil Dupras
33d9569427 Refactoring: Created hscommon.desktop
This unit hosts previously awkward UI view methods which weren't related
to the view itself, but to the current desktop environment. These
functions are now at their appropriate place.
2013-10-12 13:54:13 -04:00
Virgil Dupras
78c3c8ec2d Improved dev docs 2013-08-20 22:52:43 -04:00
Virgil Dupras
be8efea081 Fixed folder scanning in SE, which was completely broken
Oops
2013-08-18 20:50:31 -04:00
Virgil Dupras
7e8f9036d8 Began serious code documentation effort
Enabled the autodoc Sphinx extension and started adding docstrings to
classes, methods, etc.. It's quickly becoming quite interesting...
2013-08-18 18:36:09 -04:00
Virgil Dupras
e1f532e2fd Fixed broken tests 2013-08-04 09:26:18 -04:00
Virgil Dupras
1c779cb3ec Removed spurious debug code 2013-08-03 17:36:12 -04:00
Virgil Dupras
e5ce6680ca Modernized progress window GUI
Following the refactoring that has been initiated in pdfmasher's
"vala" branch, I pushed more progress window logic into the
core.

The UI code is now a bit dumber than it used to be, and the core
now directly decides when the progress window is shown and
hidden. The "job finished" notification is also directly sent by the
core. Job description update logic is handled by a core gui
textfield.

Job description contsants also moved to the core, triggering
a localisation migration from "ui" to "core".
2013-08-03 16:27:36 -04:00
Virgil Dupras
b11b97dd7c Improved delta values to support non-numerical values
Delta values now work for non-numerical values. Any column,
when its value differs from its ref, becomes orange.

A column that was already a "delta column" keeps its previous
behavior (dupe cells for these columns are always displayed in
orange).

Sorting behavior, when Dupes Only and Delta Values are enabled
at the same time, has also been extended to non-numerical
values, making it easy to mass-mark dupe rows with orange
values.

Documentation was updated, unit tests were added.

Fixes #213
2013-07-28 17:45:23 -04:00
Virgil Dupras
7891fb5396 Refactoring: Moved some code from app.DupeGuru to fs.File.
Moved DupeGuru._get_display_info() to File.get_display_info().
This method used none of the app's global state or methods
and had nothing to do there.
2013-07-14 17:43:58 -04:00
Virgil Dupras
db0901b1de Handle OSError during symlink support check
Under a windows that supports symlinks (Vista+), we still need
proper privileges. If we don't have it, OSError is raised and we
need to correctly handle this case. Ref #220.
2013-07-14 13:59:03 -04:00
Virgil Dupras
9225697053 Added hardlink/symlink support for Windows Vista+.
Fixes #220.
2013-07-14 11:58:49 -04:00
Virgil Dupras
e86b23259c Improved selection handling during ref-swapping under dupes-only mode.
Previously, our selection would simply be lost (because the dupes that were
just swapped wouldn't be present in the table). Now, instead of trying to
preserve our dupe-selection, we preserve our index-selection when in dupes-only
mode. [#222]
2013-04-28 16:27:45 -04:00
Virgil Dupras
70e505ad92 Tweaked Make Selected into Reference.
Having dupes from ref folders (which makes ref switching impossible) would make
the new feature glitchy (selection would be emptied). Now, in cases where the action
results in nothing being changed, the selection stays intact. [#222]
2013-04-28 14:12:08 -04:00
Virgil Dupras
6dddcb1a47 Fixed scanner_test which was broken.
The monkeypatching made to hscommon.io wasn't correctly transferred to Path
after the migration at [dfc82cd].
2013-04-28 10:43:20 -04:00
Virgil Dupras
4a8ce9b6c4 Updated copyright year to 2013. 2013-04-28 10:35:51 -04:00
Virgil Dupras
d4e6632e7e Fixed typo in messages.
Sucessfully --> Successfully. [#216 state:fixed]
2013-04-28 10:20:59 -04:00
Virgil Dupras
0ced3e39c8 Allow "Open selected" to open more than one file at once.
When there's mpre than 10 selected dupes, a warning is shown, asking
confirmation. [#142 state:fixed]
2013-04-28 10:12:25 -04:00
Virgil Dupras
592eba9eaa [#222] Tweaked dupe selection after a ref-swapping action. 2013-04-27 10:08:38 -04:00
Virgil Dupras
8c36218150 In addition to EnvironmentError, catch UnicodeEncodeError when performing actions on marked duplicates.
When running dupeGuru under Linux with a messed up locale, it's
UnicodeEncodeError we get. Instead of popping a reportable traceback, it's
better to just pop the Problems dialog up.
2012-10-28 11:06:09 -04:00
Virgil Dupras
111edc3ce5 Fixed a bug causing groups with more than one ref file in it to appear (which looks weird and messes with selection).
Contents scans already weeded them out, bu t they were still possible with name-based scans. Now, the Scanner removes them all.
2012-08-09 11:16:06 -04:00
Virgil Dupras
df30a31782 Refactoring: Began to phase out to the use of hscommon.io in favor of Path methods. 2012-08-09 10:53:24 -04:00