Fix negative bitmapset member not allowed in decompression #2222
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
svenklemm
requested review from
mfreed,
pmwkaa and
k-rus
and removed request for
a team
Codecov Report
@@ Coverage Diff @@
## master #2222 +/- ##
==========================================
+ Coverage 90.04% 90.15% +0.10%
==========================================
Files 211 211
Lines 34291 34240 -51
==========================================
- Hits 30879 30868 -11
+ Misses 3412 3372 -40
Continue to review full report at Codecov.
|
svenklemm
force-pushed
the
negative_bitmapset
branch
from
d3c9770 to
cd11bd9
Compare
pmwkaa
approved these changes
Aug 17, 2020
|
While its possible to create a test i don't think it's reasonable as we would have to make postgres consume 2 billion oids to guarantee we get a relid that triggers the bug |
gayyappan
approved these changes
Aug 17, 2020
Due to a typo in the code the relid of the relation instead of the rangetable index was used when building the pathkey for the DecompressChunk node. Since relids are unsigned but bitmapsets use signed int32 this lead to 'negative bitmapset member not allowed' being thrown as error when the relid is greater than INT32_MAX. This patch also adds an assert to prevent this from happening again.
svenklemm
force-pushed
the
negative_bitmapset
branch
from
cd11bd9 to
6256ea7
Compare
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 24, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2116 Propagate privileges from hypertables to chunks * timescale#2150 Lock dimension slice tuple when scanning * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix negative bitmapset member not allowed in decompression **Thanks** * @akamensky for reporting an issue with drop_chunks * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
Merged
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 25, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2116 Propagate privileges from hypertables to chunks * timescale#2150 Lock dimension slice tuple when scanning * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2256 Fix segfault in chunk_append with space partitioning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 26, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2116 Propagate privileges from hypertables to chunks * timescale#2150 Lock dimension slice tuple when scanning * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2256 Fix segfault in chunk_append with space partitioning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 26, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2116 Propagate privileges from hypertables to chunks * timescale#2150 Lock dimension slice tuple when scanning * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2256 Fix segfault in chunk_append with space partitioning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 26, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2255 Propagate privileges from hypertables to chunks * timescale#2256 Fix segfault in chunk_append with space partitioning * timescale#2259 Fix recursion in cache processing * timescale#2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2255 Propagate privileges from hypertables to chunks * timescale#2256 Fix segfault in chunk_append with space partitioning * timescale#2259 Fix recursion in cache processing * timescale#2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2255 Propagate privileges from hypertables to chunks * timescale#2256 Fix segfault in chunk_append with space partitioning * timescale#2259 Fix recursion in cache processing * timescale#2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * #2059 Improve infering start and stop arguments from gapfill query * #2067 Support moving compressed chunks * #2068 Apply SET TABLESPACE for compressed chunks * #2090 Fix index creation with IF NOT EXISTS for existing indexes * #2092 Fix delete on tables involving hypertables with compression * #2164 Fix telemetry installed_time format * #2184 Fix background worker scheduler memory consumption * #2222 Fix `negative bitmapset member not allowed` in decompression * #2255 Propagate privileges from hypertables to chunks * #2256 Fix segfault in chunk_append with space partitioning * #2259 Fix recursion in cache processing * #2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * #2059 Improve infering start and stop arguments from gapfill query * #2067 Support moving compressed chunks * #2068 Apply SET TABLESPACE for compressed chunks * #2090 Fix index creation with IF NOT EXISTS for existing indexes * #2092 Fix delete on tables involving hypertables with compression * #2164 Fix telemetry installed_time format * #2184 Fix background worker scheduler memory consumption * #2222 Fix `negative bitmapset member not allowed` in decompression * #2255 Propagate privileges from hypertables to chunks * #2256 Fix segfault in chunk_append with space partitioning * #2259 Fix recursion in cache processing * #2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Due to a typo in the code the relid of the relation instead of the
rangetable index was used when building the pathkey for the
DecompressChunk node. Since relids are unsigned but bitmapsets
use signed int32 this lead to 'negative bitmapset member not allowed'
being thrown as error when the relid is greater than INT32_MAX.
This patch also adds an assert to prevent this from happening again.