buffer: construct Buffers in JS #2866

trevnorris · 2015-09-14T22:35:12Z

Overall construction time is faster in JS, but the problem is zero-fill
of memory. Get around this by using a flag in the ArrayBuffer::Allocator
to trigger when memory should or shouldn't be zero-filled.

R=@bnoordhuis
R=@indutny

Warning. There is some mess in here. Throwing it up for general review.

indutny · 2015-09-14T22:36:54Z

lib/buffer.js

@@ -14,11 +15,19 @@ Buffer.poolSize = 8 * 1024;
 var poolSize, poolOffset, allocPool;


+binding.setupBufferJS(Buffer.prototype, bindingObj);
+const flags = bindingObj.flags;


What do you think about making this one-byte Uint8Array instead, and passing it to the C++?

nvm, you did it this way :)

indutny · 2015-09-15T04:39:43Z

lib/buffer.js

-  allocPool = binding.create(poolSize);
+  flags[kNoZeroFill] = 1;
+  allocPool = new Uint8Array(poolSize);
+  allocPool.__proto__ = Buffer.prototype;


What do you think about getting rid of this thing and using class.. extends?

super() cannot be optimized. using class.. extends was the first approach I took when reimplementing Buffer and it had significant overhead. Which is the initial reason I went the route of the C++ construction.

Overall construction time of Typed Arrays is faster in JS, but the problem with using it normally is zero-fill of memory. Get around this by using a flag in the ArrayBuffer::Allocator to trigger when memory should or shouldn't be zero-filled. Remove Buffer::Create() as it is no longer called. The creation of the Uint8Array() was done at each callsite because at the time of this patch there was a performance penalty for centralizing the call in a single function.

indutny · 2015-09-15T19:23:58Z

lib/buffer.js

-  allocPool = binding.create(poolSize);
+  flags[kNoZeroFill] = 1;
+  allocPool = new Uint8Array(poolSize);
+  Object.setPrototypeOf(allocPool, Buffer.prototype);


Hahaha, this one is better! :)

Are you afraid of class...extends? We can address it later, though.

class is too slow. The call to super() can't be optimized by v8. I learned this b/c the initial implementation used class...extends and I never saw the constructor show up in IRHydra.

You sure about this? I have seen the disassembly output, and it looked like the Buffer constructor was inlined and was called Uint8Array constructor directly

This was when class was very first supported. So additional optimizations may have been done since then.

indutny · 2015-09-15T19:25:08Z

One naming nit, otherwise LGTM.

Overall construction time of Typed Arrays is faster in JS, but the problem with using it normally is zero-fill of memory. Get around this by using a flag in the ArrayBuffer::Allocator to trigger when memory should or shouldn't be zero-filled. Remove Buffer::Create() as it is no longer called. The creation of the Uint8Array() was done at each callsite because at the time of this patch there was a performance penalty for centralizing the call in a single function. PR-URL: #2866 Reviewed-By: Fedor Indutny <fedor@indutny.com>

trevnorris · 2015-09-15T22:21:22Z

Suggested name changed. Landed in 74178a5.

Overall construction time of Typed Arrays is faster in JS, but the problem with using it normally is zero-fill of memory. Get around this by using a flag in the ArrayBuffer::Allocator to trigger when memory should or shouldn't be zero-filled. Remove Buffer::Create() as it is no longer called. The creation of the Uint8Array() was done at each callsite because at the time of this patch there was a performance penalty for centralizing the call in a single function. PR-URL: #2866 Reviewed-By: Fedor Indutny <fedor@indutny.com>

Notable changes: * buffer: - Buffers are now created in JavaScript, rather than C++. This increases the speed of buffer creation (Trevor Norris) nodejs#2866. - `Buffer#slice()` now uses `Uint8Array#subarray()` internally, increasing `slice()` performance (Karl Skomski) nodejs#2777. * fs: - `fs.utimes()` now properly converts numeric strings, `NaN`, and `Infinity` (Yazhong Liu) nodejs#2387. - `fs.WriteStream` now implements `_writev`, allowing for super-fast bulk writes (Ron Korving) nodejs#2167. * http: Fixed an issue with certain `write()` sizes causing errors when using `http.request()` (Fedor Indutny) nodejs#2824. * npm: Upgrade to version 2.14.3, see https://github.com/npm/npm/releases/tag/v2.14.3 for more details (Kat Marchán) nodejs#2822. * src: V8 cpu profiling no longer erroneously shows idle time (Oleksandr Chekhovskyi) nodejs#2324. * v8: Lateral upgrade to 4.5.103.33 from 4.5.103.30, contains minor fixes (Ali Ijaz Sheikh) nodejs#2870. - This fixes a previously known bug where some computed object shorthand properties did not work correctly (nodejs#2507). Refs: nodejs#2844 PR-URL: nodejs#2889

Notable changes: * buffer: - Buffers are now created in JavaScript, rather than C++. This increases the speed of buffer creation (Trevor Norris) #2866. - `Buffer#slice()` now uses `Uint8Array#subarray()` internally, increasing `slice()` performance (Karl Skomski) #2777. * fs: - `fs.utimes()` now properly converts numeric strings, `NaN`, and `Infinity` (Yazhong Liu) #2387. - `fs.WriteStream` now implements `_writev`, allowing for super-fast bulk writes (Ron Korving) #2167. * http: Fixed an issue with certain `write()` sizes causing errors when using `http.request()` (Fedor Indutny) #2824. * npm: Upgrade to version 2.14.3, see https://github.com/npm/npm/releases/tag/v2.14.3 for more details (Kat Marchán) #2822. * src: V8 cpu profiling no longer erroneously shows idle time (Oleksandr Chekhovskyi) #2324. * v8: Lateral upgrade to 4.5.103.33 from 4.5.103.30, contains minor fixes (Ali Ijaz Sheikh) #2870. - This fixes a previously known bug where some computed object shorthand properties did not work correctly (#2507). Refs: #2844 PR-URL: #2889

MylesBorins · 2015-11-16T19:07:16Z

landed in lts-v4.x-staging as 7df018a

indutny reviewed Sep 14, 2015
View reviewed changes

brendanashworth added the buffer Issues and PRs related to the buffer subsystem. label Sep 14, 2015

trevnorris force-pushed the avoid-zero-fill branch from ff32546 to feee1d4 Compare September 14, 2015 23:13

indutny reviewed Sep 15, 2015
View reviewed changes

trevnorris force-pushed the avoid-zero-fill branch from feee1d4 to 6dbb278 Compare September 15, 2015 18:16

trevnorris force-pushed the avoid-zero-fill branch from 6dbb278 to d235fe2 Compare September 15, 2015 18:27

indutny reviewed Sep 15, 2015
View reviewed changes

trevnorris closed this Sep 15, 2015

trevnorris added the land-on-v4.x label Sep 15, 2015

trevnorris deleted the avoid-zero-fill branch September 15, 2015 22:28

Fishrock123 mentioned this pull request Sep 16, 2015

Release proposal: v4.1.0 #2844

Closed

7 tasks

rvagg mentioned this pull request Sep 22, 2015

Release proposal: v4.1.1 #2995

Merged

jasnell added the lts-landed-on-v4.x label Oct 7, 2015

MylesBorins removed the land-on-v4.x label Nov 16, 2015

ronag mentioned this pull request Jul 14, 2019

stream: always invoke end callback #28687

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

buffer: construct Buffers in JS #2866

buffer: construct Buffers in JS #2866

trevnorris commented Sep 14, 2015

indutny Sep 14, 2015

indutny Sep 14, 2015

indutny Sep 15, 2015

trevnorris Sep 15, 2015

indutny Sep 15, 2015

trevnorris Sep 15, 2015

indutny Sep 15, 2015

trevnorris Sep 15, 2015

indutny commented Sep 15, 2015

trevnorris commented Sep 15, 2015

MylesBorins commented Nov 16, 2015

buffer: construct Buffers in JS #2866

buffer: construct Buffers in JS #2866

Conversation

trevnorris commented Sep 14, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

indutny commented Sep 15, 2015

trevnorris commented Sep 15, 2015

MylesBorins commented Nov 16, 2015