Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zlib: decompression throw on truncated input #2595

Closed
wants to merge 1 commit into from

Conversation

jhamhader
Copy link
Contributor

Check for unexpected end-of-stream error when decompressing.
If the output buffer still has space after decompressing and deflate
returned Z_OK or Z_BUF_ERROR - that means unexpected end-of-stream.
Added test-zlib-truncated.js for the case of truncated input.
Fixed the zlib dictionary test to not end the inflate stream on a
truncated output (no crc) of deflate
See issue #2043

@targos targos added the zlib Issues and PRs related to the zlib subsystem. label Aug 28, 2015
@Fishrock123
Copy link
Member


var common = require('../common');
var assert = require('assert');
var zlib = require ('zlib');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use const for values that won't be changed that you either added or modified in this commit. :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the input :)
Should I use 'let' for the scope-specific variables or is 'var' ok?

@jhamhader
Copy link
Contributor Author

Committed 'let' and 'const' for 'test-zlib-dictionary.js' and 'test-zlib-truncated.js'

@indutny
Copy link
Member

indutny commented Sep 24, 2015

cc @nodejs/collaborators

@trevnorris
Copy link
Contributor

Is this done elsewhere (as in, in other programs), and/or is it what the user would expect to happen?

@jhamhader
Copy link
Contributor Author

  • gnu gunzip: fails with unexpected end of file
  • python2 gzip: fails with IOError: CRC check failed 0xca600aac != 0x6fe58c67L
  • python3 gzip: fails with EOFError: Compressed file ended before the end-of-stream marker was reached
  • ruby gzip: fails with Zlib::GzipFile::Error: unexpected end of file
  • perl gzip: fails with unexpected end of file at (eval 15)[/usr/share/perl5/core_perl/perl5db.pl:737] line 2.

@trevnorris
Copy link
Contributor

Seems at the minimum we should be propagation an error. So will it throw for sync and emit error on async? (not familiar w/ much of the zlib api)

@jhamhader
Copy link
Contributor Author

Throw for sync.
What do you mean by emit? The callback of the async function will be called with err on the first argument.

Check for unexpected end-of-file error when decompressing.
If the output buffer still has space after decompressing and deflate
returned Z_OK or Z_BUF_ERROR - that means unexpected end-of-file.
Added test-zlib-truncated.js for the case of truncated input.
Fixed the zlib dictionary test to not end the inflate stream on a
truncated output (no crc) of deflate
@trevnorris
Copy link
Contributor

The callback of the async function will be called with err on the first argument.

Nm me. Was thinking about using zlib w/ the streams API and how a pipe would propagate the error.

@jhamhader
Copy link
Contributor Author

Sure :)
Anything else missing for the PR?

@trevnorris
Copy link
Contributor

Not from me. Would like @indutny to sign off on this as well.

@thefourtheye
Copy link
Contributor

I would prefer @chrisdickinson to sign off as well.

@jhamhader
Copy link
Contributor Author

@indutny @chrisdickinson could you please take a look and confirm?
Thanks :)

@trevnorris
Copy link
Contributor

@jhamhader ping me tomorrow if @indutny hasn't responded.

@jhamhader
Copy link
Contributor Author

Pinging

@trevnorris
Copy link
Contributor

Thanks. Let's go ahead and land this. I'll be back at my computer on Monday.

Also, I'm initially inclined to say this is semver-major. Even if it's not, don't know if this is something LTS would accept. We can check after it's in master. :-)

@Fishrock123 Fishrock123 added the semver-major PRs that contain breaking changes and should be released in the next major version. label Oct 10, 2015
@Fishrock123 Fishrock123 added this to the 5.0.0 milestone Oct 10, 2015
@rvagg rvagg mentioned this pull request Oct 16, 2015
13 tasks
@Fishrock123
Copy link
Member

ping @trevnorris

@trevnorris
Copy link
Contributor

Landed on 0fc0902. Thanks much!

@trevnorris trevnorris closed this Oct 19, 2015
trevnorris pushed a commit that referenced this pull request Oct 19, 2015
Check for unexpected end-of-file error when decompressing. If the output
buffer still has space after decompressing and deflate returned Z_OK or
Z_BUF_ERROR - that means unexpected end-of-file. Added
test-zlib-truncated.js for the case of truncated input. Fixed the zlib
dictionary test to not end the inflate stream on a truncated output (no
crc) of deflate

Fixes: #2043
PR-URL: #2595
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
rvagg pushed a commit that referenced this pull request Oct 21, 2015
Check for unexpected end-of-file error when decompressing. If the output
buffer still has space after decompressing and deflate returned Z_OK or
Z_BUF_ERROR - that means unexpected end-of-file. Added
test-zlib-truncated.js for the case of truncated input. Fixed the zlib
dictionary test to not end the inflate stream on a truncated output (no
crc) of deflate

Fixes: #2043
PR-URL: #2595
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
@rvagg rvagg mentioned this pull request Oct 21, 2015
rvagg added a commit to rvagg/io.js that referenced this pull request Oct 29, 2015
Notable changes:

* buffer: (Breaking) Removed both 'raw' and 'raws' encoding types from Buffer,
  these have been deprecated for a long time (Sakthipriyan Vairamani) nodejs#2859.
* console: (Breaking) Values reported by console.time() now have 3 decimals of
  accuracy added (Michaël Zasso) nodejs#3166.
* fs:
  - fs.readFile*(), fs.writeFile*(), and fs.appendFile*() now also accept a file
    descriptor as their first argument (Johannes Wüller) nodejs#3163.
  - (Breaking) In fs.readFile(), if an encoding is specified and the internal
    toString() fails the error is no longer thrown but is passed to the callback
    (Evan Lucas) nodejs#3485.
  - (Breaking) In fs.read() (using the fs.read(fd, length, position, encoding,
    callback) form), if the internal toString() fails the error is no longer
    thrown but is passed to the callback (Evan Lucas) nodejs#3503.
* http:
  - Fixed a bug where pipelined http requests would stall (Fedor Indutny) nodejs#3342.
  - (Breaking) When parsing HTTP, don't add duplicates of the following headers:
    Retry-After, ETag, Last-Modified, Server, Age, Expires. This is in addition
    to the following headers which already block duplicates: Content-Type,
    Content-Length, User-Agent, Referer, Host, Authorization,
    Proxy-Authorization, If-Modified-Since, If-Unmodified-Since, From, Location,
    Max-Forwards (James M Snell) nodejs#3090.
  - (Breaking) The callback argument to OutgoingMessage#setTimeout() must be a
    function or a TypeError is thrown (James M Snell) nodejs#3090.
  - (Breaking) HTTP methods and header names must now conform to the RFC 2616
    "token" rule, a list of allowed characters that excludes control characters
    and a number of separator characters. Specifically, methods and header names
    must now match /^[a-zA-Z0-9_!#$%&'*+.^`|~-]+$/ or a TypeError will be thrown
    (James M Snell) nodejs#2526.
* node:
  - (Breaking) Deprecated the _linklist module (Rich Trott) nodejs#3078.
  - (Breaking) Removed require.paths and require.registerExtension(), both had
    been previously set to throw Error when accessed
    (Sakthipriyan Vairamani) nodejs#2922.
* npm: Upgraded to version 3.3.6 from 2.14.7, see
  https://github.com/npm/npm/releases/tag/v3.3.6 for more details. This is a
  major version bump for npm and it has seen a significant amount of change.
  Please see the original npm v3.0.0 release notes for a list of major changes
  (Rebecca Turner) nodejs#3310.
* src: (Breaking) Bumped NODE_MODULE_VERSION to 47 from 46, this is necessary
  due to the V8 upgrade. Native add-ons will need to be recompiled
  (Rod Vagg) nodejs#3400.
* timers: Attempt to reuse the timer handle for setTimeout().unref(). This fixes
  a long-standing known issue where unrefed timers would perviously hold
  beforeExit open (Fedor Indutny) nodejs#3407.
* tls:
  - Added ALPN Support (Shigeki Ohtsu) nodejs#2564.
  - TLS options can now be passed in an object to createSecurePair()
    (Коренберг Марк) nodejs#2441.
  - (Breaking) The default minimum DH key size for tls.connect() is now 1024
    bits and a warning is shown when DH key size is less than 2048 bits. This a security consideration to prevent "logjam" attacks. A new minDHSize TLS
    option can be used to override the default. (Shigeki Ohtsu) nodejs#1831.
* util:
  - (Breaking) util.p() was deprecated for years, and has now been removed
    (Wyatt Preul) nodejs#3432.
  - (Breaking) util.inherits() can now work with ES6 classes. This is considered
    a breaking change because of potential subtle side-effects caused by a
    change from directly reassigning the prototype of the constructor using
    `ctor.prototype = Object.create(superCtor.prototype, { constructor: { ... } })`
    to using `Object.setPrototypeOf(ctor.prototype, superCtor.prototype)`
    (Michaël Zasso) nodejs#3455.
* v8: (Breaking) Upgraded to 4.6.85.25 from 4.5.103.35 (Ali Ijaz Sheikh) nodejs#3351.
  - Implements the spread operator, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
    for further information.
  - Implements new.target, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target
    for further information.
* zlib: Decompression now throws on truncated input (e.g. unexpected end of
  file) (Yuval Brik) nodejs#2595.

PR-URL: nodejs#3466
rvagg added a commit that referenced this pull request Oct 29, 2015
Notable changes:

* buffer: (Breaking) Removed both 'raw' and 'raws' encoding types from Buffer,
  these have been deprecated for a long time (Sakthipriyan Vairamani) #2859.
* console: (Breaking) Values reported by console.time() now have 3 decimals of
  accuracy added (Michaël Zasso) #3166.
* fs:
  - fs.readFile*(), fs.writeFile*(), and fs.appendFile*() now also accept a file
    descriptor as their first argument (Johannes Wüller) #3163.
  - (Breaking) In fs.readFile(), if an encoding is specified and the internal
    toString() fails the error is no longer thrown but is passed to the callback
    (Evan Lucas) #3485.
  - (Breaking) In fs.read() (using the fs.read(fd, length, position, encoding,
    callback) form), if the internal toString() fails the error is no longer
    thrown but is passed to the callback (Evan Lucas) #3503.
* http:
  - Fixed a bug where pipelined http requests would stall (Fedor Indutny) #3342.
  - (Breaking) When parsing HTTP, don't add duplicates of the following headers:
    Retry-After, ETag, Last-Modified, Server, Age, Expires. This is in addition
    to the following headers which already block duplicates: Content-Type,
    Content-Length, User-Agent, Referer, Host, Authorization,
    Proxy-Authorization, If-Modified-Since, If-Unmodified-Since, From, Location,
    Max-Forwards (James M Snell) #3090.
  - (Breaking) The callback argument to OutgoingMessage#setTimeout() must be a
    function or a TypeError is thrown (James M Snell) #3090.
  - (Breaking) HTTP methods and header names must now conform to the RFC 2616
    "token" rule, a list of allowed characters that excludes control characters
    and a number of separator characters. Specifically, methods and header names
    must now match /^[a-zA-Z0-9_!#$%&'*+.^`|~-]+$/ or a TypeError will be thrown
    (James M Snell) #2526.
* node:
  - (Breaking) Deprecated the _linklist module (Rich Trott) #3078.
  - (Breaking) Removed require.paths and require.registerExtension(), both had
    been previously set to throw Error when accessed
    (Sakthipriyan Vairamani) #2922.
* npm: Upgraded to version 3.3.6 from 2.14.7, see
  https://github.com/npm/npm/releases/tag/v3.3.6 for more details. This is a
  major version bump for npm and it has seen a significant amount of change.
  Please see the original npm v3.0.0 release notes for a list of major changes
  (Rebecca Turner) #3310.
* src: (Breaking) Bumped NODE_MODULE_VERSION to 47 from 46, this is necessary
  due to the V8 upgrade. Native add-ons will need to be recompiled
  (Rod Vagg) #3400.
* timers: Attempt to reuse the timer handle for setTimeout().unref(). This fixes
  a long-standing known issue where unrefed timers would perviously hold
  beforeExit open (Fedor Indutny) #3407.
* tls:
  - Added ALPN Support (Shigeki Ohtsu) #2564.
  - TLS options can now be passed in an object to createSecurePair()
    (Коренберг Марк) #2441.
  - (Breaking) The default minimum DH key size for tls.connect() is now 1024
    bits and a warning is shown when DH key size is less than 2048 bits. This a security consideration to prevent "logjam" attacks. A new minDHSize TLS
    option can be used to override the default. (Shigeki Ohtsu) #1831.
* util:
  - (Breaking) util.p() was deprecated for years, and has now been removed
    (Wyatt Preul) #3432.
  - (Breaking) util.inherits() can now work with ES6 classes. This is considered
    a breaking change because of potential subtle side-effects caused by a
    change from directly reassigning the prototype of the constructor using
    `ctor.prototype = Object.create(superCtor.prototype, { constructor: { ... } })`
    to using `Object.setPrototypeOf(ctor.prototype, superCtor.prototype)`
    (Michaël Zasso) #3455.
* v8: (Breaking) Upgraded to 4.6.85.25 from 4.5.103.35 (Ali Ijaz Sheikh) #3351.
  - Implements the spread operator, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
    for further information.
  - Implements new.target, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target
    for further information.
* zlib: Decompression now throws on truncated input (e.g. unexpected end of
  file) (Yuval Brik) #2595.

PR-URL: #3466
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-major PRs that contain breaking changes and should be released in the next major version. zlib Issues and PRs related to the zlib subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants