Commit graph

507 commits

Author SHA1 Message Date
skybldev
2e3b292c26 server: improve migration, camera now UnicodeText
- [server/migration] merge add_camera -> add_date_taken
- [server/migration] rename add_date_taken -> add_date_taken_and_camera
- [server/migration] add_date_taken_and_camera: iterate through posts in db
  instead of files
- [server] changed Post.camera from sa.Text -> sa.UnicodeText
- [server] improved stability of `func/images.py`
- [todo] will add migration that fixes image dimensions and regenerates
  thumbnails later
2022-01-10 16:41:21 -05:00
skybldev
aa03eaba44 server: metadata.py -> images.py [WIP, broken]
[server] `func/images.py`:
  - `Image` is now `Image` and `Video`, with their own post-type-specific
    operations; some of them from `func/metadata.py`. as a result, post
    type checking will need to be done outside of the classes
  - apply orientation to [fixed] thumbnail generation and image conversion
  - moved `_execute` to a global function; it now purely uses streams
    instead of tempfiles
  - `func/images.py::Video::to_mp4()` removed code that downscales resolution
    to an even number since ffmpeg does it automatically
  - use fstrings instead of .format and % in certain places
  - DEPRECATING FLASH FILES; there really is no place for them in a
    photo-oriented booru such as this. will slowly phase it out overtime.
    added an error message for it
  - changed EXIF orientation reference URL
[server] `func/posts.py`:
  - `update_post_content()`: updated to use `func/images.py`'s new
    features
2022-01-06 18:31:25 -05:00
skybldev
a4ea05a0e4 server: fix swapped dimensions on some images
- [overview] this commit fixes an issue where some images would be rendered
  stretched in a browser due to a swapped width and height because of
  intentional EXIF orientation mechanics.
- [server] `func/metadata.py`: added function to resolve the image's
  dimensions after taking the EXIF orientation into consideration. normally,
  this would only be done on the client side; however, since the server takes
  those values into consideration for its own operations, it could also be
  considered a "client". for example, thumbnail generation also depends on the
  post's dimensions.
- [server] `func/posts.py`: refactored `update_post_content`. `func/images.py`
  will no longer be used to determine the dimensions of a file since
  `func/metadata.py` is now responsible for that.
- [server] refactored `func/metadata.py`
- [TODO] create migration to correct post dimensions
- [TODO] merge `func/metadata.py` with `func/images.py` and refactor
- [TODO] fix thumbnail generation
2022-01-01 17:38:23 -05:00
skybldev
40331f2ba2 server: fixed tag typo in metadata.py 2021-12-03 00:02:36 -05:00
skybldev
efe217344d server: add post camera attr and functionality
- [overview] add camera attribute to posts. parsing is done in
  `func/metadata.py` and takes any of the available tags corresponding
  to "make" and "model" properties and concatenates them into a string
- [server] improved `func/metadata.py`:
  - added camera resolve functions for photos and videos
  - moved ffmpeg subprocess and exif image opening to separate function
  - optionally reuse existing collection of extracted tags in any of the
    functions
  - iterative approach to checking for tags' existence as opposed to
    imperative
  - (somewhat) better error handling
- [server] created alembic migration in `adb2acef2492_add_camera.py`
  - not only adds columns, but also scans files and updates their camera
    string
- [server] added camera attribute functionality and improved error
  handling in `func/posts.py`
- [server] add camera attribute to `model/post.py`
2021-12-02 21:27:26 -05:00
skybldev
a48c792322 server: catch failure to read image exif
- [server] wrapped exif image reading in a try..except, since it can
  fail for some images. Also added a logger message to make it easier to
  debug in the future, to fix compatibility issues and make debugging
  easier
2021-11-29 21:23:47 -05:00
skybldev
066993ee07 server: implemented filter/sort by date taken
- [overview] implemented filtering and sorting by date taken, optionally
  with null.
- [server] added `SORT_DESC_NULL_LAST` search token in `search/tokens.py`,
  `search/base_search_config.py`
- [server] implemented `SORT_DESC_NULL_LAST` functionality in
  `search/executor.py`
- [server] added date taken search aliases in
  `search/configs/post_search_config.py`
- [server] added `NullCriterion` in `search/criteria.py`; it's literally
  null
- [server] implemented filtering by null for date criterion in
  `search/configs/util.py`
- [server] aliased `NullCriterion` to "null", "none", "unknown", or "?" in
  `search/parser.py`
- [server][minor] added "posted" and "edited" aliases to creation_date
  and last_edit_date
2021-11-29 17:04:56 -05:00
skybldev
f73dd02c7d server: attempt to use DateTimeOriginal
- [server] attempt to use DateTimeOriginal in `func/metadata.py` to
  resolve date taken (compatibility issue originally found in photo
  taken on LG Optimus L90)
2021-11-29 13:41:03 -05:00
skybldev
1bcf7e73ad server: added date taken detection for videos
- [server] added date taken detection for videos in `func/metadata.py`
  using subprocess and ffmpeg
2021-11-28 23:18:43 -05:00
skybldev
f9af3afdf8 server: added basic date taken functionality
- [server] created basic utility for resolving date taken @ `func/metadata.py`
- [server] utilize `func/metadata.py` in `func/posts.py`
- [server] created Alembic migration that adds `date_taken` column to `post`
- [server] updated `model/post.py:Post` to have new column
2021-11-28 12:04:09 -05:00
skybldev
19b1ec1496 server/deps: add exif as dependency 2021-11-28 12:00:07 -05:00
Shyam Sunder
d083084407 server/tests: use transactional db for faster unit tests
* `test_modify_saves_non_empty_diffs` needs non-transactional
  db, so moved to seperate file
* Replaced incompatable usage of `db.session.rollback()`
  with parametrerized function calls
* xfail conditionals for search removed, as we can no longer
  get current driver with binds
* Also remove usage of deprecated `pytest.yield_fixture`
2021-09-23 12:24:56 -04:00
Shyam Sunder
ad9d3599bc server/net: return more useful error messages 2021-09-22 22:08:07 -04:00
Shyam Sunder
c3b81371d8 client+server/docker: fix ARM build platform issue 2021-09-19 12:03:32 -04:00
Shyam Sunder
4f57f49ebe client+server: migrate to GitHub actions 2021-09-19 11:01:47 -04:00
Shyam Sunder
e4a253fd25 client+server: fixed style errors 2021-09-13 13:25:37 -04:00
neobooru
fa4997fbb9 server: fix issue where no video files could be uploaded 2021-06-07 00:37:30 +02:00
neobooru
f497dca92f server: update docker image base to alpine:3.13
We do this so that we don't have to use 'edge' packages, which aren't (always) ABI compatible
2021-06-01 18:20:51 +02:00
neobooru
5ea9e27e48 Merge branch 'avif'
Merges PR #399
2021-06-01 16:57:29 +02:00
Ruin0x11
a2b68925ac Support BMP format uploads 2021-05-09 01:29:36 -07:00
Ruin0x11
2949431d9a Add libheif/libavif to Dockerfile dependencies 2021-05-07 22:25:59 -07:00
Ruin0x11
7e27df835c Add AVIF/HEIF/HEIC upload support 2021-05-07 21:20:42 -07:00
Ruin0x11
169593ea36 Add AVIF/HEIC detection
ffmpeg doesn't support HEIC decoding yet...
2021-05-07 14:36:58 -07:00
nothink (Satoshi Ishii)
535aa0d8fe
Suppressed the use of SQLAlchemy 1.4 2021-04-20 22:52:29 +09:00
Shyam Sunder
545b5828b5 server/func/mime: support ftypM4V file signature 2021-03-30 09:52:49 -04:00
Shyam Sunder
2b9a4ab786 server/net: prevent youtube-dl errors when downloading image links 2021-01-07 08:28:22 -05:00
Shyam Sunder
c732e62844 server/net: fix error handling 2021-01-06 10:37:59 -05:00
Shyam Sunder
c7461c7f65 server/net: improve youtube-dl functionality, enforce size limits 2021-01-05 17:05:57 -05:00
Shyam Sunder
2dfd1c2192 server/search: add MD5-based search 2021-01-05 13:51:39 -05:00
Shyam Sunder
2bdb072296 server/posts: store and provide MD5 checksums 2021-01-05 13:20:01 -05:00
Shyam Sunder
58678b4504 server/func/mailer: Attempt to manually start TLS for SMTP
Fixes #365
2020-12-02 14:01:43 -05:00
Shyam Sunder
a302b2c4a4 server: enable large file support in database 2020-10-11 12:50:21 -04:00
Shyam Sunder
143f633eaa server/func/webhooks: call webhooks asynchronously 2020-10-06 11:55:09 -04:00
Shyam Sunder
697bd45420 server/tag-categories: sort responses by order 2020-09-24 22:50:28 -04:00
Shyam Sunder
d4f72de8c2 server/tests: fix failing tests 2020-09-24 19:09:54 +02:00
neobooru
d2b6ecef4d server+client: update tag category api + fix formatting 2020-09-23 13:48:47 +02:00
neobooru
368372e36d server/tests: fix failing tests 2020-09-20 12:07:42 +02:00
neobooru
06ad8b1882 client+server: add tag category ordering feature
Fixes  #209
2020-09-19 22:55:17 +02:00
Shyam Sunder
1ef0419dc2 server/pools: serialize pools as micro resource within post resources
Fixes #348
2020-09-19 10:29:09 -04:00
Shyam Sunder
0dd427755b client+server: fix linter issues due to updated pre-commit hooks 2020-09-01 14:07:39 -04:00
Shyam Sunder
e656a3c46a server/docker: unify test and main Dockerfiles 2020-08-28 14:43:10 -04:00
Shyam Sunder
4595f9a2aa server: API support for webhooks
Closes #339
2020-08-13 22:41:43 -04:00
Shyam Sunder
57193b5715 client+server: implement code autoformatting using prettier and black 2020-06-06 08:58:23 -04:00
Shyam Sunder
c06aaa63af dev: add pre-commit hooks for pytest and docker building 2020-06-05 12:47:23 -04:00
Shyam Sunder
c0d0c4c894 client+server: normalize trailing newlines 2020-06-05 10:54:32 -04:00
Shyam Sunder
ea623449e7 server: format code to flake8 2020-06-05 10:02:18 -04:00
Shyam Sunder
c5358f7f83 client+server: add post pools feature 2020-06-04 21:01:28 -04:00
Shyam Sunder
48c9001194 server/docker: include setuptools in installation 2020-06-04 18:51:30 -04:00
Shyam Sunder
ea675d20cb server/docker: fix missing installation requirements
Furthermore, an update to Pillow has improved the floating-point
precision of the image hash algorithm, requiring minor updates to
the respective unit tests.

See https://github.com/python-pillow/Pillow/pull/4320
2020-06-04 16:38:26 -04:00
Shyam Sunder
b0f1b8c230 fix python lint issues 2020-06-03 11:55:50 -04:00