Updating ultrajson#1
Open
WilliamBinquet wants to merge 815 commits into
Open
Conversation
Update github-actions
Re: PyCQA/isort#2078 Committed via https://github.com/asottile/all-repos
The stdlib json module has supported these since Python 3.4. Since `enum.IntEnum` is an `int` subclass (and hence `PyLong_*` work as expected), ujson has supported them for a long time as well, but this was so far not tested.
Add test for int- and float-derived Enums
Both of these licenses require the copyright notice and the license text to be distributed in all copies. Instead of merely mentioning them, include the full text in LICENSE.txt Fixes #565; see that issue for further analysis.
…ed-repository_url Replace deprecated repository_url with repository-url
Update pypa/cibuildwheel action to v2.12.1
updates: - [github.com/psf/black: 22.12.0 → 23.3.0](psf/black@22.12.0...23.3.0) - [github.com/pre-commit/pygrep-hooks: v1.9.0 → v1.10.0](pre-commit/pygrep-hooks@v1.9.0...v1.10.0)
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [mheap/github-action-required-labels](https://togithub.com/mheap/github-action-required-labels) | action | major | `v3` -> `v4` | | [pypa/cibuildwheel](https://togithub.com/pypa/cibuildwheel) | action | patch | `v2.12.1` -> `v2.12.3` | --- ### Release Notes <details> <summary>mheap/github-action-required-labels</summary> ### [`v4`](https://togithub.com/mheap/github-action-required-labels/releases/tag/v4) [Compare Source](https://togithub.com/mheap/github-action-required-labels/compare/v3...v4) Tag that always points to the latest commit in the v4.x.x series of releases </details> <details> <summary>pypa/cibuildwheel</summary> ### [`v2.12.3`](https://togithub.com/pypa/cibuildwheel/releases/tag/v2.12.3) [Compare Source](https://togithub.com/pypa/cibuildwheel/compare/v2.12.2...v2.12.3) - 🐛 Fix an import error when running on Python 3.7. ([#​1479](https://togithub.com/pypa/cibuildwheel/issues/1479)) ### [`v2.12.2`](https://togithub.com/pypa/cibuildwheel/releases/tag/v2.12.2) [Compare Source](https://togithub.com/pypa/cibuildwheel/compare/v2.12.1...v2.12.2) - 🐛 Fix a bug that caused an extra empty config-setting to be passed to the backend when CIBW_BUILD_FRONTEND is set to `build`. ([#​1474](https://togithub.com/pypa/cibuildwheel/issues/1474)) - 🐛 Fix a crash that occurred when overwriting an existing wheel on Windows. ([#​1464](https://togithub.com/pypa/cibuildwheel/issues/1464)) - 🛠 Pinned version updates, including CPython 3.10.11, 3.11.3, pip 23.1 and wheel 0.40.0. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on the first day of the month" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/ultrajson/ultrajson). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42Ni4xIiwidXBkYXRlZEluVmVyIjoiMzUuNjYuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->
](https://renovatebot.com){kind=link}
PyIter_Check() is used to skip straight to the container types when type-forking during serialisation. However, PyIter_Check() only returns true for generators – not arbitrary iterables – and since we don't allow generators to be used in place of lists, this shortcut is never used. (If it did work, then it would also break strings since they are iterable but not to be treated as containers.)
All strings are encoded with surrogatepass so they can always be converted to UTF-8. Only memory errors can reach *PyUnicodeToUTF8Raw() failed* code path.
decode_string() writes to an `escBuffer` (ds->escStart to ds->escEnd). `escBuffer` is initialised as a 256 x JSUINT32 array on the stack but can be replaced with a larger buffer malloc()ed on the heap. There is code to resize it up again but it's never used. The code is pretty unintelligible. This is what I think it does: * ds->start to ds->end represent the current position within JSON payload (start of a string excluding the leading ") and the end of payload respectively * ds->escHeap initialises to false (per parser – not reset if parsing a second string within one ujson.loads() call) * `escBuffer` (ds->escStart to ds->escEnd) is initialised as a 256 x JSUINT32 array * If ds->end - ds->start (remaining bytes to parse in whole JSON) is larger than escBuffer: - resize escBuffer to ds->end - ds->start using non-heap resize path - set ds->escHeap to true * If ds->end - ds->start (remaining bytes to parse in whole JSON) is later becomes larger than escBuffer again, using the *on the heap resize* path However, the number of remaining bytes to parse in a JSON payload will always decrease and escBuffer will never shrink so that last condition can never happen.
Buffer_EscapeStringUnvalidated() has no failure mode. Any code handling its return code is unreachable.
If given null as its buffer input, it'll create its own. But we never do.
…ools_scm] section (#746)
The code is currently split into pure C code (/lib) and the Python bindings (/python). This sounds nice until you actually look at it and realise just how much this separation costs. * Most exceptions are OverflowError because the code that raises exceptions can't reference true exception types provided by Python.h * Python functions have to be wrapped in trivial callback functions passed around as void pointers in structs so that the encode/decoder can avoid directly referencing the Python functions * void pointer anarchy Now: lib/ultrajsondec.c + python/JSONtoObj.c -> decode.c lib/ultrajsonenc.c + python/objToJSON.c -> encode.c Changes beyond the move/concatenation are as minimal as I could make it (deduplicating #includes, adjusting paths, shuffling function prototypes around).
* #include <stdbool.h> * Drop the sporadically used TRUE/FALSE #defines * Replace everywhere integers are used as booleans with bools
* Anywhere it was being used as a bucket for storing functions, use the functions directly * Anywhere it was used to store decoder state, move that information to the DecoderState parent struct
Without the /lib vs /python split, said functions can just be invoked directly. This makes it much easier to follow where the code is going.
So they're easier to recognise
Formerly, JSONObjectEncoder->prv was overloaded to hot-swap between storing the default() function and (presumably?) the previous encoder. This was so confusing that it hid the fact that there is only ever one JSONObjectEncoder() per serialisation so both JSONObjectEncoder->prv and JSONTypeContext->encoder_prv were just copying nulls around the rest of the time and can be safely removed.
Another relic of the /lib vs /python split that we no longer have...
Strictly speaking, we should propagate the const-ness around but it creates a horrendous function colouring game (made worse by the assumption that all the PFN_PyTypeToJSON functions have the same signature) that we don't gain anything from playing. The JSON encode path has no reason to start writing to its input strings so I don't really care if we loose that compiler assertion. Closes #713.
The formerly missing stdint.h support (which we already depend on) was added in MSVC 2010. We no longer need compatibility aliases for uint64_t et al. Also replace JSUTF16 and JSUTF32 aliases which are actually used to represent UTF-8.
This appears to have been left over from before ultrajson became a Python library. We don't expect people to link against ujson so it makes no sense to export symbols.
* Move JSONTypeContext out of the shared header into encode.c where it's used * Set the proper type for JSONTypeContext's void* prv attribute (I still can't tell what prv is short for – "previous" would be my guess until I looked at the code) to TypeContext * This lets us drop the GET_TC(tc) macro in favour of just tc->prv (with all the type casting to TypeContext* it used to do no longer required) * This in turn revealed that tc->prv and pc are the same thing (something the surrounding code seemed to have forgotten) and the pc references can be removed * Which then showed that there's never more than one TypeContext so JSONTypeContext and TypeContext can be merged into a single struct
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.