tag:blogger.com,1999:blog-43564022235817680632024-03-18T10:48:17.324+01:00Java is the new CHigh performance Java, realtime distributed computing, other stuff.Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.comBlogger25125tag:blogger.com,1999:blog-4356402223581768063.post-81213899982111103872017-12-31T00:32:00.002+01:002017-12-31T00:32:26.309+01:00A fast React Live Editing implementation<h2 style="clear: both; text-align: center;">
A fast React Live Editing implementation</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.juptr.io/imgcache/hosted/moru0011/9ccba060-e725-4185-a34d-1905bbfc7a5b.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="250" data-original-width="800" height="199" src="https://www.juptr.io/imgcache/hosted/moru0011/9ccba060-e725-4185-a34d-1905bbfc7a5b.jpg" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px;">Hi folks, part of my personal mission to integrate good ol' Java with modern JavaScript frameworks was the implementation of "Hot Reload". I am not talking of "automatic Browser refresh" but of live patching the code of a React client without losing component state.</span><br style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px;" /><br />
<a href="https://www.juptr.io/juptrblogs/_57034b5b-fa76-4ef3-b6a8-7c715ee4f1a8.html">This blog has moved, continue reading here</a>Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com88tag:blogger.com,1999:blog-4356402223581768063.post-87135737682486309052017-08-15T22:18:00.001+02:002017-08-15T22:18:38.062+02:00Developing a React+JSX app from Java without node, webpack, and babel <br />
<div class="elem" id="main-image" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div class="noemph" style="border-left: 12px solid transparent;">
</div>
</div>
<br />
<div class="elem" style="orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; widows: 2;">
<div class="noemph editable juptr-abstract-editor" style="border-left: 12px solid transparent; line-height: 1.6em; margin: 0px; padding: 0px;">
<div class="style-scope juptr-abstract-editor editable" style="line-height: 1.6em; margin: 0px; padding: 0px;">
<div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px;">
Implementing a web app server side using Java has been getting ugly recently as its required to run a a full nodejs stack including a load of npm packages and javascript related stuff. By implementing a bundling and JSX Transpilation in native Java, kontraktor can help out and make things simple again (#MTSA).</div>
<div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px;">
<br /></div>
<div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px;">
<img height="360" src="https://www.juptr.io/imgcache/hosted/moru0011/775dc7e4-e93e-4bf7-8f71-38212129035d.jpg" width="640" /></div>
<div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px;">
</div>
<div class="elem" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div class="noemph editable juptr-paragraph-editor" style="border-left: 12px solid transparent; font-size: 18px; line-height: 1.6em; margin: 0px; padding: 0px;">
<div class="style-scope juptr-paragraph-editor editable" style="font-size: 18px; line-height: 1.6em; margin: 0px; padding: 0px;">
</div>
</div>
</div>
<div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px;">
</div>
<div class="elem" id="main-image" style="orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; widows: 2;">
<div class="noemph" style="border-left: 12px solid transparent;">
<div class="style-scope juptr-image-editor editable" id="sidediv" style="display: inline-block; padding: 8px 0px; vertical-align: top; width: 688.002px;">
<div align="center" class="fullbleed style-scope juptr-image-editor editable" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; margin: 0px; padding: 8px 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
you can't have a java article without picturing a cup of coffee ..</div>
Gone are the days of server side web apps, though there are apparently still people out there trying to get something fancy out of .. [this blog has moved, <span style="font-size: large;"><a href="https://www.juptr.io/juptrblogs/_e67b0d1e-98f8-44fb-b431-845005f36129.html">continue reading here</a>]</span><br />
<br /><div align="center" class="fullbleed style-scope juptr-image-editor editable" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-style: italic; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; margin: 0px; padding: 8px 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<br /></div>
</div>
</div>
</div>
</div>
</div>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com76tag:blogger.com,1999:blog-4356402223581768063.post-69920450992213386372017-02-25T02:24:00.001+01:002017-02-25T02:25:30.245+01:00Move Code Not Data: 'Sandboxing' using Remote Lambda Execution<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.juptr.io/imgcache/hosted/moru0011/42d7e787-370f-45e2-9a7a-e9f391f9654e.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="372" src="https://www.juptr.io/imgcache/hosted/moru0011/42d7e787-370f-45e2-9a7a-e9f391f9654e.jpg" width="640" /></a></div>
<br />
<div style="text-align: center;">
<span style="background-color: white; color: #333333; font-family: "roboto" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 18px; font-style: italic;">Leveraging remote lambda execution to avoid being IO bound in a data intensive </span><strike style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">micro</strike><span style="background-color: white; color: #333333; font-family: "roboto" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 18px; font-style: italic;">service system. </span></div>
<div style="text-align: left;">
<span style="background-color: white; color: #333333; font-family: "roboto" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 18px; font-style: italic;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "roboto" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 18px; font-style: italic;"><br /></span></div>
<div style="text-align: left;">
<span style="color: #333333; font-family: "roboto" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="background-color: white; font-size: 18px;">This blog has moved. <a href="https://www.juptr.io/juptrblogs/a2b6139f-b273-42bd-b533-e4695e27e353.html" target="_blank">Read the full article here</a>.</span></span></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com19tag:blogger.com,1999:blog-4356402223581768063.post-3126230702117470412016-11-29T19:19:00.002+01:002016-11-29T19:19:39.581+01:00Practice Report: A webapp with Polymer.js<img height="300" src="https://www.juptr.io/imgcache/hosted/moru0011/02515a19-157f-4396-94a0-612281f21ee3.jpg" width="640" /><br />
<br />
<br />
<span style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">Part </span><b style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">two</b><span style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;"> of a series of our journey building </span><a href="https://www.juptr.io/" style="background-color: white; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;" target="_blank">juptr.io</a><span style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">:</span><br style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;" /><span style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">Our experiences building a front end with google's </span><b style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">Polymer.js</b><br />
<b style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;"><br /></b>
<div class="elem" style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif;">
<div class="noemph editable juptr-subtitle-editor" style="border-left: 12px solid transparent; font-size: 24px; font-weight: bold; margin: 0px; padding: 20px 4px 12px 0px;">
<div class="style-scope juptr-subtitle-editor editable" style="padding: 20px 4px 12px 0px;">
Why Polymer ?</div>
</div>
</div>
<div class="elem" style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif;">
<div class="noemph editable juptr-paragraph-editor" style="border-left: 12px solid transparent; font-size: 18px; line-height: 1.6em; margin: 0px; padding: 0px;">
<div class="style-scope juptr-paragraph-editor editable" style="line-height: 1.6em; padding: 0px;">
</div>
There are numerous alternatives when choosing a framework to build web front ends some of most popular probably being ... <a href="https://www.juptr.io/juptrblogs/_50500fd7-1a89-468c-8347-eff21789d52b.html" style="color: black; font-style: italic; text-decoration: none;" target="_blank">[this blog has moved, continue reading here]</a></div>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com16tag:blogger.com,1999:blog-4356402223581768063.post-73773211198668566952016-11-12T18:11:00.000+01:002016-11-12T18:11:12.448+01:00Make distributed programming great again: Microservices with Distributed Actor Systems<div class="style-scope juptr-abstract-editor editable" style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; line-height: 1.6em; padding: 4px;">
Part one of a series of our journey building juptr.io: </div>
<div style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">
How a fundamental abstraction streamlines, simplifies and speeds up developing a web scale system</div>
<div style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">
<br /></div>
<div style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">
<img height="370" src="https://www.juptr.io/imgcache/hosted/moru0011/b560430c-fc40-483a-a93d-018d615d705e.jpg" width="640" /></div>
<div style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic;">
</div>
<div class="elem editable juptr-paragraph-editor" style="font-size: 18px; line-height: 1.6em; margin: 0px; padding: 0px;">
<div class="style-scope juptr-paragraph-editor editable" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: normal; letter-spacing: normal; line-height: 1.6em; margin: 0px; orphans: 2; padding: 0px; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
</div>
</div>
<br />
<div class="elem" id="main-image">
<div class="style-scope juptr-image-editor editable" id="sidediv" style="display: inline-block; font-size: 16px; font-style: italic; padding: 8px 0px; vertical-align: top; width: 700px;">
<div align="center" class="fullbleed style-scope juptr-image-editor editable" style="font-size: 16px; font-style: italic; margin: 0px; padding: 8px 0px;">
goes higher, faster and avoids dirty feet :)</div>
<div class="fullbleed style-scope juptr-image-editor editable" style="font-size: 16px; font-style: italic; margin: 0px; padding: 8px 0px; text-align: left;">
<span data-auto-link="true" data-href="http://Juptr.io" style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: normal;"><a href="http://juptr.io/" target="_blank">Juptr.io</a></span><span style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: normal;"> is a content personalization, sharing and authoring platform. We crawl 10'th of thousands of blogs and media sites (german + english), then classify the documents to enable personalized content curation, consumption & discussion.</span></div>
<div style="background-color: white; color: #333333; font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px; font-style: normal;">
This requires<br />
<div>
<ul>
<li>to crawl, store, query, analyze and classify millions of documents</li>
<li>horizontal scalability</li>
<li>a failure resistant distributed (micro)services architecture</li>
<li>java, javascript interoperability</li>
</ul>
Compared to ordinary enterprise-level projects, a startup has much tighter time and budget constraints (aka startup in europe), so reduction of development effort was a major priority.<br />
We decided to use the following foundation stack .. <a href="https://www.juptr.io/juptrblogs/c7839e9a-da8b-4a60-bc66-94558c30c4da.html" style="color: black; font-style: italic; text-decoration: none;" target="_blank">[this blog has moved, continue reading here]</a></div>
</div>
</div>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com20tag:blogger.com,1999:blog-4356402223581768063.post-77021993984561991672016-10-19T21:53:00.003+02:002016-10-19T21:53:42.407+02:00<div class="elem editable juptr-title-editor" style="background-color: white; color: #333333; font-family: Roboto, 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 40px; font-weight: 900; padding-bottom: 18px; padding-top: 32px;">
<div class="style-scope juptr-title-editor editable" style="padding-bottom: 18px; padding-top: 32px;">
Word disambiguation combining LDA and word embeddings</div>
</div>
<div class="elem editable juptr-abstract-editor" style="background-color: white; color: #333333; font-family: Roboto, 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; line-height: 1.6em; margin: 0px; padding: 4px;">
<div class="style-scope juptr-abstract-editor editable" style="line-height: 1.6em; padding: 4px;">
<b>Topical search</b> tries to take into account the occurence of search-term-related words to avoid matching irrelevant documents. This post lines out how LDA can be used to disambiguate a word-set obtained using word embeddings. </div>
<div class="style-scope juptr-abstract-editor editable" style="line-height: 1.6em; padding: 4px;">
<img src="https://www.juptr.io/imgcache/hosted/moru0011/036c27c2-6d07-4bc7-9fc1-6d4017e5cbf1.jpg" /></div>
<div class="style-scope juptr-abstract-editor editable" style="line-height: 1.6em; padding: 4px;">
<a href="https://www.juptr.io/juptrblogs/5c39f627-1338-43aa-8e6b-f5b987ab4a15.html" target="_blank">[this blog has moved, continue reading here]</a></div>
<div class="style-scope juptr-abstract-editor editable" style="line-height: 1.6em; padding: 4px;">
<br /></div>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com15tag:blogger.com,1999:blog-4356402223581768063.post-12941312898890543382016-10-19T21:51:00.006+02:002016-10-19T21:51:48.229+02:00<div class="elem editable juptr-title-editor" style="background-color: white; color: #333333; font-family: Roboto, 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 40px; font-weight: 900; padding-bottom: 18px; padding-top: 32px;">
<div class="style-scope juptr-title-editor editable" style="padding-bottom: 18px; padding-top: 32px;">
Texting Bots: The command line interface rebranded ?</div>
</div>
<br />
<div class="elem" id="main-image" style="background-color: white; color: #333333; font-family: Roboto, 'Helvetica Neue', Helvetica, Arial, sans-serif;">
</div>
<br />
<div class="elem editable juptr-abstract-editor" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Roboto, 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 18px; font-style: italic; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 1.6em; margin: 0px; orphans: auto; padding: 4px; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px;">
<div class="style-scope juptr-abstract-editor editable" style="font-size: 18px; font-style: italic; line-height: 1.6em; margin: 0px; padding: 4px;">
Texting Bots and Chat Bots are all the hype, but is typing to a mostly dumb AI the way users want to interact with software ? Maybe other aspects besides AI and texting are the real killer feature of chat bots.</div>
<div class="style-scope juptr-abstract-editor editable" style="font-size: 18px; font-style: italic; line-height: 1.6em; margin: 0px; padding: 4px;">
<img height="383" src="https://www.juptr.io/imgcache/hosted/moru0011/2e1e9b2a-1448-4f03-82e4-b18f0881964b.jpg" width="640" /></div>
<div class="style-scope juptr-abstract-editor editable" style="font-size: 18px; font-style: italic; line-height: 1.6em; margin: 0px; padding: 4px;">
[<a href="https://www.juptr.io/juptrblogs/0fbd479f-23ed-4483-8819-95112ea5e665.html" target="_blank">blog has moved, continue here</a>]</div>
<div class="style-scope juptr-abstract-editor editable" style="font-size: 18px; font-style: italic; line-height: 1.6em; margin: 0px; padding: 4px;">
<br /></div>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com13tag:blogger.com,1999:blog-4356402223581768063.post-7478814760318494832015-11-20T13:11:00.001+01:002015-11-20T20:55:47.727+01:00Clarification and Fix for Serialization ExploitThere have been reports of a Java serialization based epxloit (e.g. <a href="http://www.infoq.com/news/2015/11/commons-exploit" rel="nofollow" target="_blank">here</a>), enabling the attacker to execute arbitrary code. As usual, there is a lot of half baked reasoning about this issue, so I'd like to present a fix for my fast-serialization library as well as tweaks to block deserialization of certain classes for stock JDK serialization.<br />
<br />
In my view the issue is not located in Java serialization, but insecure programming in some common open source libraries.<br />
<br />
<blockquote class="tr_bq">
<i><span style="font-size: large;">Attributing this exploit to Java Serialization is wrong, the exploit requires door opening non-jdk code to be present on the server.</span></i></blockquote>
<br />
Unfortunately some libraries accidentally do this, however its very unlikely this is a wide spread (mis-) programming pattern.<br />
<br />
<br />
<span style="font-size: large;"><b>How can a class at server side open the door ?</b></span><br />
<br />
<br />
<ul>
<li>the class has to define readObject() </li>
<li>the readObject() method has to be implemented in a way that allows to execute code sent from remote</li>
</ul>
<br />
I struggle to imagine a sane use case on why one would do something along the line of<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">"readObject(ObjectInputStream in) { Runtime.exec(in.readString()); }"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">or</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">"readObject(ObjectInputStream in) { define_and_execute_class(in.readBytes()); }"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
Anyway, it happened. Again: something like the code above must be present on the class path at server side, it cannot be injected from outside.<br />
<br />
<br />
<br />
<b><span style="font-size: large;">Fixing this for <a href="https://github.com/RuedigerMoeller/fast-serialization" target="_blank">fast-serialization</a></span></b><br />
<br />
<br />
<ul>
<li>fast-serialization 2.43 introduces a delegate interface, which enables blacklisting/whitelisting of classes, packages or whole package trees. By narrowing down classes being allowed for serialization, one can assure only expected objects are deserialized.</li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3Mr-s_X-lt1ovc2SH0sr5xwFC7-p76AtqOZGSvBuoas8asehug93cbU3Vw9AHQFRgs-oD3KXpzWmaMhO8E5pQqHJbUxr-GAOKT3atfSFkRPkZTp-S2WJhj0fBkP9wqNOrY6VHNBwLil8/s1600/Screenshot+from+2015-11-20+12%253A47%253A48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3Mr-s_X-lt1ovc2SH0sr5xwFC7-p76AtqOZGSvBuoas8asehug93cbU3Vw9AHQFRgs-oD3KXpzWmaMhO8E5pQqHJbUxr-GAOKT3atfSFkRPkZTp-S2WJhj0fBkP9wqNOrY6VHNBwLil8/s1600/Screenshot+from+2015-11-20+12%253A47%253A48.png" /></a></div>
<br />
<br />
<ul>
<li>if you cannot upgrade to fst 2.43 for backward compatibility reasons, at least register a custom serializer for the problematic classes (see exploit details). As registered custom serializers are processed by fast-serialization <b>before</b> falling back to JDK serialization emulation, throwing an exception in the <b>read</b> or <b>instantiate</b> method of the custom serializer will effectively block the exploit.</li>
<li>There is no significant performance impact as this is executed on initialization only</li>
</ul>
<br />
<br />
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;">Hotfixing stock JDK Serialization implementation</span></b><br />
<br />
(Verfied for JDK 1.8 only)<br />
<br />
Use a subclass of ObjectInputStream instead of the original implementation and put in a blacklisting/whitelisting by overriding this method in ObjectInputStream:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF2thAYSXBnPPZVAXjshYFDL2fp-KsADONFfnToEUctDFfNjSLhFLs20T4h8tEtsJOBTPqN-TreBtMkBFpOGgE5HUDHP7XsQkweHYdai0ONrqjHMY2rMClOarBiqVNbirzuGlbiQabOWg/s1600/Screenshot+from+2015-11-20+12%253A49%253A18.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF2thAYSXBnPPZVAXjshYFDL2fp-KsADONFfnToEUctDFfNjSLhFLs20T4h8tEtsJOBTPqN-TreBtMkBFpOGgE5HUDHP7XsQkweHYdai0ONrqjHMY2rMClOarBiqVNbirzuGlbiQabOWg/s1600/Screenshot+from+2015-11-20+12%253A49%253A18.png" /></a></div>
<br />
<br />
(maybe also check <span style="background-color: #344134; color: #a9b7c6; font-family: "source code pro";"><b>resolveProxyClass</b></span>, I'm not sure wether this also can get exploited [probably not]).<br />
This will have a performance impact, so its crucial to have a efficient implementation of blacklisting/whitelisting here.<br />
<br />
<br />
<br />
<br />Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com20tag:blogger.com,1999:blog-4356402223581768063.post-44853187420884120852015-07-31T00:24:00.002+02:002015-08-04T20:15:00.056+02:00Polymer WebComponents Client served by Java ActorsFront end development productivity has been stalling for a long time. I'd even say regarding productivity, there was no substantial improvement since the days of Smalltalk (Bubble Event Model, MVC done right, component based).<br />
<br />
Weird enough, for a long time things got worse: Document centric web technology made classic and proven component centric design patterns hard to implement.<br />
<br />
For reasons unknown to me (fill in some major raging), it took some 15 years until a decent and well thought webcomponent standard (proposal) arised: <b><a href="http://webcomponents.org/">WebComponents</a></b>. Its kind an umbrella of several standards, most importantly:<br />
<ul>
<li>Html Templates - the ability to have markup snippets which are not "seen" and processed by the browser but can be "instantiated" programmatically later on.</li>
<li>Html Imports - finally the ability to include a bundle of css, html and javascript implicitely resolving dependencies by not loading an import twice. Style does not leak out to the main document.</li>
<li>Shadow Dom</li>
</ul>
It would not be a real web technology without some serious drawbacks: Html Imports are handled by the browser, so you'll get many http requests when loading a webcomponent app using html imports to resolve dependencies.<br />
As the Html standard is strictly separated from the Http protocol, simple solutions such as server side imports (to reduce number of http requests) can never be part of an official standard.<br />
<br />
Anyway, with Http-2's pipelining and multiplexing features html-imports won't be an issue within some 3 to 5 years (..), until then some workarounds and shim's are required.<br />
<br />
<b>Github repo </b>for this blog "Buzzword Cloud": <a href="https://github.com/RuedigerMoeller/kontraktor-polymer-example" target="_blank"><b><span style="color: #0b5394;">kontraktor-polymer-example</span></b></a><br />
<br />
<b>Live Demo</b>: <a href="http://46.4.83.116/polymer/" target="_blank"><b><span style="color: #0b5394;">http://46.4.83.116/polymer/</span></b></a> (just sign in with anything except user 'admin'). Requires Websocket connectivity, modern browser (uses webcomps-lite.js, so no IE)<b>. </b>Curious wether it survives ;-).<br />
<b><br /></b>
Currently Chrome and Opera implement WebComponents, however there is a well working polyfill for mozilla + safari and a last resort polyfill for ie. <br />
Using a server side html import shim (as provided by kontraktor's http4k), web component based apps run on android 4.x and IOS safari.<br />
<br />
<b><span style="font-size: large;">(<a href="https://www.polymer-project.org/1.0/">Polymer</a>) WebComponents in short</span></b><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJcwuWybiLFFQf6ycGI5K2h7B9WmZd7ch-0PJ3hyphenhyphenqmUpOPjWBiUamO4xoKyiqbz34xKSJ2jDWeio5jBAWunMCeLbm7gDONZS9dSIQXpF5mzP_Dm-9neb5wxhk4RGzDs-QlzNVW35rD8HQ/s1600/webcomp.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJcwuWybiLFFQf6ycGI5K2h7B9WmZd7ch-0PJ3hyphenhyphenqmUpOPjWBiUamO4xoKyiqbz34xKSJ2jDWeio5jBAWunMCeLbm7gDONZS9dSIQXpF5mzP_Dm-9neb5wxhk4RGzDs-QlzNVW35rD8HQ/s1600/webcomp.png" /></a></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: center;">
<i>a web component with dependencies (</i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">link rel='import'</span><i>), embedded style, a template and code</i></div>
<br />
<b><br /></b>
<b>Dependency Management with Imports ( "<link rel='import' ..>" )</b><br />
<br />
The browser does not load an imported document twice and evaluates html imports linearly. This way dependencies are resolved in correct order implicitly. An imported html snippet may contain (nearby) arbitrary html such as scripts, css, .. most often it will contain the definition of a web component.<br />
<br />
<b>Encapsulation</b><br />
<b><br /></b>
Styles and templates defined inside an imported html component do not "leak" to the containing document.<br />
Web components support data binding (one/two way). Typically a Web component coordinates its direct children only (encapsulation). Template elements can be easily accessed with "this.$.elemId".<br />
<div>
<br /></div>
<b>Application structure</b><br />
<b><br /></b>
An application also consists of components. Starting from the main-app component, one subdivides an app hierarchically into smaller subcompontents, which has a nice self-structuring effect, as one creates reusable visual components along the way.<br />
<br />
The main <b>index.html</b> typically refers to the main app component only (kr-login is an overlay component handling authentication with kontraktor webapp server):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUsE2sB0qyQkubySbd0TEXGOOv6biMzVCisg9MEXFISuMiCyE1M0LARr8nf6MQDhyphenhyphen3dDerwCLXDGpLTRis3SMwZkJArQYpPxf4ht2NNvQsLrbuVJebsCNaOHvzJLdgkfybQFCOF0vZSds/s1600/app.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUsE2sB0qyQkubySbd0TEXGOOv6biMzVCisg9MEXFISuMiCyE1M0LARr8nf6MQDhyphenhyphen3dDerwCLXDGpLTRis3SMwZkJArQYpPxf4ht2NNvQsLrbuVJebsCNaOHvzJLdgkfybQFCOF0vZSds/s1600/app.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
That's nice and pretty straight forward .. but lets have a look what my simple sample application's footprint looks like:<br />
<br />
<div style="text-align: center;">
<span style="font-size: x-large;"><b>Web Reality:</b></span></div>
<div style="text-align: center;">
<span style="font-size: x-large;"><b><br /></b></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif9AMnnkqBRFW6OT3WmewAHtxVDpwuhaKwe-Xf7DB3Iz-tGaogTqVYHcSiA65K0HWqJ78GnE3ucWzORsOPh37dcBMtbQADnhhJGGbx-fYoe-ZL9qHX2U5i3R9v4cSJSbP4QHVbPdd8Uwc/s1600/chrome.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif9AMnnkqBRFW6OT3WmewAHtxVDpwuhaKwe-Xf7DB3Iz-tGaogTqVYHcSiA65K0HWqJ78GnE3ucWzORsOPh37dcBMtbQADnhhJGGbx-fYoe-ZL9qHX2U5i3R9v4cSJSbP4QHVbPdd8Uwc/s1600/chrome.png" /></a></div>
<br />
hm .. you might imagine how such an app will load on a mobile device, as the number of concurrent connections typically is limited to 2..6 and a request latency of 200 to 500ms isn't uncommon. As bandwidth increases continously, but latency roughly stays the same reducing the number of requests pays off even at cost of increased initial bandwidth for many apps.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
In order to reduce the number of requests and bandwidth usage, the JavaScript community maintains a load of tools minifying, aggregating and preprocessing resources. When using Java at the server side, one ends up having two build systems, gradle or maven for your java stuff, node.js based javascript tools like bower, grunt, vulcanize etc. . In addition a lot of temporary (redundant) artifacts are created this way.<br />
<br />
As Java web server landscape mostly sticks to server-centric "Look-'ma-we-<u><b>can</b></u>-do-ze-web<b style="background-color: white; color: #252525; font-family: sans-serif; font-size: 14px; line-height: 12.8000001907349px;">™"</b> applications, its hardly possible to make use of modern javascript frameworks using Java at server side, especially as REAL JAVA DEVELOPERS DO NOT SIMPLY INSTALL NODE.JS <span style="font-size: x-small;">(though they have a windows installer now ;) ..)</span>. Nashorn unfortunately isn't yet there, currently it fails to replace node.js due to missing API or detail incompatibilities.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7k0BxR-J6jOn6EKPKJEwVQPIRFHPZoGHjp9Q9nV7cou3GwJLcazSI-KLs4UBq-5H49nfxM8SMapSs03r4FheF8NXRCbzo9efLxvUibR0fEF8j1ZaXP-qy5_9iVL7wh-ici5jb9Ab3eLw/s1600/hero.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="211" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7k0BxR-J6jOn6EKPKJEwVQPIRFHPZoGHjp9Q9nV7cou3GwJLcazSI-KLs4UBq-5H49nfxM8SMapSs03r4FheF8NXRCbzo9efLxvUibR0fEF8j1ZaXP-qy5_9iVL7wh-ici5jb9Ab3eLw/s320/hero.jpg" width="320" /></a><br />
<br />
<br />
<br />
I think its time for an unobtrusive pointer to my kontraktor actor library, which abstracts away most of the messy stuff allowing for direct communication of JavaScript and Java Actors via http or websockets.<br />
<br />
<br />
<br />
Even when serving single page applications, there is stuff only a web server can implement effectively:<br />
<br />
<ul>
<li>dynamically inline Html-Import's </li>
<li>dynamically minify scripts (even inline javascript)</li>
</ul>
<br />
In essence inlining, minification and compression are temporary wire-level optimizations, no need to add this to the build and have your project cluttered up. Kontraktor's Http4k optimizes served content dynamically if run in production mode.<br />
The same application with (kontraktor-)Http4k in production mode:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgviBYujZaBAtiHdbhBv6QQudXyYfjq7RbHs5uVXOJ_LnUyKl125_11jA2MoMaltQQFxvEbpmXybmVaAstJhx80EeY8y6H8QybESxyPWU9VnhBKimI9bH2kMre7THvCeJPVSdq4adLdyQ/s1600/load_fast.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgviBYujZaBAtiHdbhBv6QQudXyYfjq7RbHs5uVXOJ_LnUyKl125_11jA2MoMaltQQFxvEbpmXybmVaAstJhx80EeY8y6H8QybESxyPWU9VnhBKimI9bH2kMre7THvCeJPVSdq4adLdyQ/s1600/load_fast.png" /></a></div>
<br />
even better: As imports are removed during server side inlining, the web app runs under Safari IOS and Android 4.4.2 default browser.<br />
<br />
<div>
<span style="font-size: large;"><b>Actor based Server side</b></span><br />
<br />
Wether its a TCP Server Socket accepting client(-socket)s or a client browser connecting a web server: its structurally the same pattern:<br />
<br />
1. server listens for authentication/connection requests.<br />
2. server authenticates/handles a newly connecting client and instantiates/opens a client connection (webapp terminology: session).<br />
<br />
What's different when comparing a TCP server and a WebApp Http server is<br />
<ul>
<li>underlying transport protocol</li>
<li>message encoding</li>
</ul>
As <a href="https://github.com/RuedigerMoeller/kontraktor" target="_blank">kontraktor</a> maps actors to a concrete transport topology at runtime, a web app server application does not look special:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSgyO-gPUzGOZDflT3-fyJ13m7CMaQiD618bz4pJT_FyBbQRMtNf6FUsTdIISHaC28xWSNc4YcMw4HV2IKQngQYqR2fGO9zIm3qgYzgfgK0Ivz3qqFUar8r9jjG-WMJsChYKd6rKC8MBs/s1600/autserver2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSgyO-gPUzGOZDflT3-fyJ13m7CMaQiD618bz4pJT_FyBbQRMtNf6FUsTdIISHaC28xWSNc4YcMw4HV2IKQngQYqR2fGO9zIm3qgYzgfgK0Ivz3qqFUar8r9jjG-WMJsChYKd6rKC8MBs/s1600/autserver2.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: center;">
<i>a generic server using actors</i></div>
<br />
The decision on transport and encoding is done by publishing the server actor. The underlying framework (<a href="https://github.com/RuedigerMoeller/kontraktor" target="_blank">kontraktor</a>) then will map the high level behaviour description to a concrete transport and encoding. E.g. for websockets + http longpoll transports it would look like:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxQZbX3oW_3w-MG-pkzc0cfBWFk9Kc5cH6r8xaCrJf8e1J-IYeog-VJuzPH0cIPbLS0nUB-92glob6-oD3taM69JVFe0G9prZSBCqbHt5FipcYSBRgyALYZMfLJh6q9UDuPX9mq55MLXI/s1600/publishapi.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxQZbX3oW_3w-MG-pkzc0cfBWFk9Kc5cH6r8xaCrJf8e1J-IYeog-VJuzPH0cIPbLS0nUB-92glob6-oD3taM69JVFe0G9prZSBCqbHt5FipcYSBRgyALYZMfLJh6q9UDuPX9mq55MLXI/s1600/publishapi.png" /></a></div>
<br />
On client side, <a href="https://github.com/RuedigerMoeller/kontraktor/tree/trunk/modules/kontraktor-http/src/main/javascript/js4k">js4k.js</a> implements the api required to talk to java actors (using a <a href="https://github.com/RuedigerMoeller/kontraktor/wiki/Kontraktor-3#js4kjs">reduced tell/ask - style API</a>).<br />
<br />
So with a different transport mapping, a webapp backend might be used as an in-process or tcp-connectable service.<br />
<br />
So far so good, however a webapp needs html-import inlining, minification and file serving ...<br />
At this point there is an end to the abstraction game, so kontraktor simply leverages the flexibility of <a href="http://undertow.io/undertow-docs/undertow-docs-1.2.0/index.html">RedHat's Undertow</a> by providing a "resource path" FileResource handler.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhC50_VaMrVZ7LDBYSVnWh09bnyDFFk0QuSrQ7EQoURoaUmFdl5RAxkHqhzhCKpOAQAgfqY_688elLPglFWC4vF_WO41MvoMC1iUhR3_lpJF_39FygE2uWecz4VtJACEoX-JOd9aBZVM-E/s1600/respath.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhC50_VaMrVZ7LDBYSVnWh09bnyDFFk0QuSrQ7EQoURoaUmFdl5RAxkHqhzhCKpOAQAgfqY_688elLPglFWC4vF_WO41MvoMC1iUhR3_lpJF_39FygE2uWecz4VtJACEoX-JOd9aBZVM-E/s1600/respath.png" /></a></div>
<br />
The resource path file handler parses .html files and inlines+minifies (depending on settings) html-import's, css links and script tags. Ofc for development this should be turned off.<br />
The resource path works like Java's classpath, which means a referenced file is looked up starting with the first path entry advancing along the resource path until a match is found. This can be nice in order to quickly switch versions of components used and keep your libs organized (no copying required).<br />
<br />
As html is fully parsed by Http4k (<a href="http://jsoup.org/">JSoup</a> parser ftw), its recommended to keep your stuff syntactically correct. In addition keep in mind that actors require non-blocking + async implementation of server side logic, have your blocking database calls "outsourced" to kontraktors thread pool like<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqdQajj75YbjvIkhq185CEFObUgGu-auk0cFBIug8hTasWxsfZqJllmGj8NV14TFLPZ0cshY8XS9j5dbdAOjD-zF7QKsRn3Gc_i-ZCEcf4XstqIx2WujzGmVALO8NXNyfmUpFfDwfk8p8/s1600/sql.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqdQajj75YbjvIkhq185CEFObUgGu-auk0cFBIug8hTasWxsfZqJllmGj8NV14TFLPZ0cshY8XS9j5dbdAOjD-zF7QKsRn3Gc_i-ZCEcf4XstqIx2WujzGmVALO8NXNyfmUpFfDwfk8p8/s1600/sql.png" /></a></div>
<br />
<br />
<b>Conclusion:</b><br />
<ul>
<li>javascript frameworks + web standards keep improving. A rich landscape of libraries and ready to use components has grown.</li>
<li>they increasingly come with node.js based tooling </li>
<li>JVM based non-blocking servers scale better and have a much bigger pool of server side software components</li>
<li>kontraktor http4k + js4k.js helps closing the gap and simplifies development by optimizing webapp file structure and size dynamically and abstracting (not annotating!) away irrelevant details and enterprisey ceremony.</li>
</ul>
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com39tag:blogger.com,1999:blog-4356402223581768063.post-41039237038419766482015-06-27T12:36:00.000+02:002015-07-04T21:39:57.707+02:00Don't REST ! Revisiting RPC performance and where Fowler might have been wrong ..[<b>Edit: </b>Title is a click bait of course, Fowler is aware of the async/sync issues, recently posted http://martinfowler.com/articles/microservice-trade-offs.html with clarifying section regarding async. ]<br />
<br />
Hello my dear citizens of planet earth ...<br />
<br />
There are many good reasons to decompose large software systems into decoupled message passing components (team size + decoupling, partial + continuous software delivery, high availability, flexible scaling + deployment architecture, ...).<br />
<br />
With distributed applications, there comes the need for ordered point to point message passing. This is different to client/server relations, where many clients send requests at low rate and the server can choose to scale using multiple threads processing requests concurrently.<br />
<blockquote class="tr_bq">
<i><b>Remote Messaging performance is to distributed systems what method invocation performance is for non-distributed monolithic applications.</b></i></blockquote>
(guess what is one of the most optimized areas in the JVM: method invocation)<br />
<div>
<br />
[Edit: with "REST", I also refer to HTTP based webservice style API, this somewhat imprecise]</div>
<br />
<b><span style="font-size: large;">Revisiting high level remoting Abstractions</span></b><br />
<span style="font-size: large;"><br /></span>
There were various attempts at building high-level, location transparent abstractions (e.g. corba, distributed objects), however in general those idea's have not received broad acceptance.<br />
<br />
This article of Martin Fowler sums up common sense pretty well:<br />
<br />
<a href="http://martinfowler.com/articles/distributed-objects-microservices.html">http://martinfowler.com/articles/distributed-objects-microservices.html</a><br />
<br />
Though not explicitely written, the article implies <b>synchronous</b> remote calls, where a sender blocks and waits for a remote result to arrive thereby including cost of a full network roundtrip for each remote call performed.<br />
<br />
With <b>asynchronous</b> remote calls, many of the complaints do not hold true anymore. When using asynchronous message passing, the granularity of remote calls is not significant anymore.<br />
<br />
"course grained" processing<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">remote.getAgeAndBirth().<b>then</b>( (age,birth) -> .. );</span></blockquote>
<br />
is not significantly faster than 2 "fine grained" calls<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"><b>all</b>( remote.getAge(), remote.getBirth() )</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"> .<b>then</b>( resultArray -> ... );</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span></blockquote>
<br />
as both variants include network <b>round trip </b>latency only<b> once</b>. <br />
<br />
On the other hand with <b>synchronous</b> remote calls, every single remote method call has a penalty of one network round trip, and <b>only then</b> Fowlers arguments hold.<br />
<br />
Another element changing the picture is the availability of "Spores", a snippet of code which can be passed over the network and executed at receiver side e.g.<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">remote.<b>doWithPerson</b>( "Heinz", heinz -> {<br /> <span style="color: #444444;">// executed remotely, stream data back to callee</span><br /> <b>stream</b>( heinz.salaries().sum() / 12 ); <b>finish</b>();<br /> }).<b>then</b>( averageSalary -> .. );</span></blockquote>
<br />
Spore's are implementable effectively with availability of VM's and JIT compilation.<br />
<br />
Actor Systems and similar asynchronous message passing approaches have gained popularity in recent years. Main motivation was to ease concurrency and the insight that multithreading with shared data does not scale well and is hard to master in an industrial grade software development environment.<br />
<br />
As large servers in essence are "distributed systems in a box", those approaches apply also for distributed systems.<br />
<br />
Following I'll test performance of remote invocations of some frameworks. I'd like to proof that established frameworks are far from what is technically possible and want to show that popular technology choices such as REST are fundamentally inept to form the foundation of large and fine grained distributed applications.<br />
<br />
<br />
<b><span style="font-size: large;">Test Participants</span></b><br />
<br />
<b>Disclaimer:</b> As tested products are of medium to high complexity, there is danger of misconfiguration or test errors, so if anybody has a (verfied) improvement to one of the testcases, just drop me a comment or file an issue to the github repository containing the tests:<br />
<br />
<a href="https://github.com/RuedigerMoeller/remoting-benchmarks">https://github.com/RuedigerMoeller/remoting-benchmarks</a>.<br />
<br />
I verified by googling forums etc. that numbers are roughly in line with what others have observed.<br />
<br />
Features I expect from a distributed application framework:<br />
<ul>
<li>Ideally fully location transparent. At least there should be a concise way (e.g. annotations, generators) to do marshalling half-automated. </li>
<li>It is capable to map responses to their appropriate request callbacks automatically (via callbacks or futures/promises or whatever).</li>
<li>its asynchronous</li>
</ul>
<br />
products tested (<b>Disclaimer:</b> I am the author of kontraktor):<br />
<ul>
<li><b>Akka 2.11</b><br />Akka provides a high level programming interface, marshalling and networking details are mostly invisible to application code (full location transparency).</li>
<li><b>Vert.x 3.1</b><br />provides a weaker level of abstraction compared to actor systems, e.g. there are no remote references. Vert.x has a symbolic notion of network communication (event bus, endpoints). <br />As it's "polyglot", marshalling and message encoding need some manual support. <br />Vert.x is kind of a platform and addresses many practical aspects of distributed applications such as application deployment, integration of popular technology stacks, monitoring, etc.</li>
<li><b>REST (RestExpress)</b><br />As Http 1.1 based REST is limited by latency (synchronous protocol), I just choosed this more or less randomly.</li>
<li><b><a href="http://ruedigermoeller.github.io/kontraktor/" target="_blank">Kontraktor 3</a></b>, distributed actor system on Java 8. I believe it hits a sweet spot regarding performance, ease of use and mind model complexity. Kontraktor provides a concise, mostly location transparent high level programming model (Promises, Streams, Spores) supporting many transports (tcp, http long poll, websockets).</li>
</ul>
<br />
Libraries skipped:<br />
<ul>
<li>finagle - requires me to clone and build their fork of thrift 0.5 first. Then I'd have to define thrift messages, then generate, then finally run it. </li>
<li>parallel universe - at time of writing the actor remoting was not in a testable state ("Galaxy" is alpha), examples are without build files, the gradle build did not work. Once i managed to build, the programs where expecting configuration files which I could not find. Maybe worth a revisit (accepting pull requests as well :) ).</li>
</ul>
<br />
<span style="font-size: large;"><b>The Test</b></span><br />
<br />
I took a standard remoting example:<br />
<blockquote class="tr_bq">
The <b>"Ask"</b> testcase: </blockquote>
<blockquote class="tr_bq">
The sender sends a message of two numbers, the remote receiver answers with the sum of those 2 numbers. The remoting layer has to track and match requests and responses as there can be tens of thousand "in-flight". </blockquote>
<blockquote class="tr_bq">
The <b>"Tell" </b>testcase: </blockquote>
<blockquote class="tr_bq">
Sender sends fire-and forget. No reply is sent from the receiver. </blockquote>
<br />
<span style="font-size: large;"><b>Results</b></span><br />
<br />
<b>Attention</b>: Don't miss notes below charts.<br />
<br />
Platform: Linux Centos 7 dual socket 20 real cores @2.5 GHZ, 64GB ram. As the tests are ordered point to point, none of the tests made use of more than 4 cores.<br />
<br />
<table border="1" cellpadding="0" cellspacing="0" dir="ltr" style="border-collapse: collapse; border: 1px solid #ccc; font-family: arial,sans,sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="176"></col><col width="100"></col><col width="100"></col></colgroup><tbody>
<tr style="height: 21px;"><td style="padding: 2px 3px 2px 3px; vertical-align: bottom;"></td><td data-sheets-value="[null,2,"tell Sum"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">tell Sum <br />
(msg/second)</td><td data-sheets-value="[null,2,"ask Sum"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">ask Sum (msg/second)</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Kontraktor Idiomatic"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">Kontraktor Idiomatic</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,1900000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1.900.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,860000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">860.000</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Kontraktor Sum-Object"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">Kontraktor Sum-Object</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,1450000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1.450.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,795000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">795.000</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Vert.x 3 "]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">Vert.x 3 </td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,200000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">200.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,200000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">200.000</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"AKKA (Kryo)"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">AKKA (Kryo)</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,120000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">120.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,65000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">65.000</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"AKKA"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">AKKA</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,70000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">70.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,64500]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">64.500</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"RestExpress"]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">RestExpress</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,15000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">15.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,15000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">15.000</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Rest 100 con."]" style="padding: 2px 3px 2px 3px; vertical-align: bottom;">Rest >15 connections</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,48000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">48.000</td><td data-sheets-numberformat="[null,2,"#,##0",1]" data-sheets-value="[null,3,null,48000]" style="padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">48.000</td></tr>
</tbody></table>
<br />
let me chart that for you ..<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrGL-TeXILi36zi9pyqZ29OfbBKnhUb-I87fBv5A44Et7ON2qkmg_Spz2neruUuO_Czu1U4sFiLAqGmQuVgT_mQaEQY_E2auwWZXpzLVUxUrRPfVo6zMbO2DQvmqtph6nzy8HkQ3xUvaQ/s1600/stats.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrGL-TeXILi36zi9pyqZ29OfbBKnhUb-I87fBv5A44Et7ON2qkmg_Spz2neruUuO_Czu1U4sFiLAqGmQuVgT_mQaEQY_E2auwWZXpzLVUxUrRPfVo6zMbO2DQvmqtph6nzy8HkQ3xUvaQ/s1600/stats.png" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<span style="font-size: large;">Remarks</span>:<br />
<br />
<ul>
<li><b>Kontraktor 3 </b>outperforms by a huge margin. I verified the test is correct and all messages are transmitted (if in doubt just clone the git repo and reproduce). </li>
<li><b>Vert.x 3 </b>seems to have a built-in rate limiting. I saw peaks of 400k messages/second however it averaged at 200k (hints for improvement welcome). In addition, the first connecting sender only gets 15k/second throughput. If I stop and reconnect, throughput is as charted. <br />I tested the very first Vert.x 3 final release. For marshalling fast-serialization (FST) was used (also used in kontraktor). Will update as Vert.x 3 matures</li>
<li><b>Akka</b>. I spent quite some time on improving the performance with mediocre results. As Kryo is roughly of same speed as fst serialization, I'd expect at least 50% of kontraktor's performance.<br /><b><br />Edit: </b>Further analysis shows, Akka is hit by poor serialization performance. It has an option to use Protobuf for encoding, which might improve results (but why kryo did not help then ?). <br /><br />Implications of using protobuf:<br />* need each message be defined in a .proto file, need generator to be run<br />* frequently additional datatransfer is done like "app data => generated messages => app data"<br />* no reference sharing support, no cyclic object graphs can be transmitted<br />* no implicit compression by serialization's reference sharing.<br />* unsure wether the ask() test profits as it did not profit from Kryo as well<br />* Kryo performance is in the same ballpark as protobuf but did not help that much either.<br /><br />Smell: I had several people contacting me aiming to improve Akka results. They disappear somehow. <br /><br />Once I find time I might add a protobuf test. Its a pretty small test program, so if there was an easy fix, it should not be a huge effort to provide it. The git repo linked above contains a maven buildable ready to use project.</li>
<li><b>REST</b>. Poor throughput is not caused by RestExpress (which I found quite straight forward to use) but by Http1.1's dependence on latency. If one moves a server to other hardware (e.g. different subnet, cloud), throughput of a service can change drastically due to different latency. This might change with Http 2.<br />Good news is: You can <use> </any> <chatty> { encoding: for messages }, as it won't make a big difference for point to point REST performance.<br />Only when opening many connections (>20) concurrently, throughput increases. This messes up transaction/message ordering, so can only be used for idempotent operations (a species mostly known from white papers and conference slides, rarely seen in the wild).</li>
</ul>
<br />
<br />
<b><span style="font-size: large;">Misc Observations</span></b><br />
<br />
<b><br /></b>
<b>Backpressure</b><br />
<b><br /></b>
Sending millions of messages as fast as possible can be tricky to implement in a non-blocking environment. A naive send loop <br />
<ul>
<li>might block the processing thread </li>
<li>build up a large outbound queue as put is faster than take+sending. </li>
<li>can prevent incoming Callbacks from being enqueued + executed (=Deadlock or OOM).</li>
</ul>
Of course this is a synthetic test case, however similar situations exist e.g. when streaming large query results or sending large blob's to other node's (e.g. init with reference data).<br />
<br />
None of the libraries (except rest) did that out of the box:<br />
<ul>
<li>Kontraktor requires a manual increase of queue sizes over default (default is 32k) in order to not deadlock in the "<b>ask</b>" test. In addition its required to programatically adopt send rate by using the backpressure signal raised by the TCP stack (network send blocks). This can be done non-blocking, "offer()" style.</li>
<li>For VertX i used a periodic task sending a burst of 500 to 1000 messages. Unfortunately the optimal number of messages per burst depends on hardware performance, so the test might need adoption when run on e.g. a Laptop.</li>
<li>For Akka I send 1 million messages each 30 seconds in order to avoid implementation of application level flow control. It just queued up messages and degrades to like 50 msg/s when used naively (big loop).</li>
<li>REST was not problematic here (synchronous Http1.1 anyway). Degraded by default.</li>
</ul>
<br />
<b><br /></b>
<br />
<span style="font-size: large;"><b>Why is kontraktor remoting that faster ?</b></span><br />
<ul>
<li>premature optimization </li>
<li>adaptive batching works wonders, especially when applied to reference sharing serialization.</li>
<li>small performance compromises stack up, reduce them bottom up </li>
</ul>
Kontraktor actually is far from optimal. It still generates and serializes a "MethodCall() { int targetId, [..], String methodName, Object args[]}" for each message remoted. It does not use Externalizable or other ways of bypassing generic (fast-)serialization.<br />
Throughputs beyond 10 million remote method invocations/sec have been proved possible at cost of a certain fragility + complexity (unique id's and distributed systems ...) + manual marshalling optimizations.<br />
<br />
<br />
<span style="font-size: large;"><b>Conclusion</b></span><br />
<br />
<ul>
<li>As scepticism regarding distributed object abstractions is mostly performance related, high performance asynchronous remote invocation is a game changer</li>
<li>Popular libraries have room for improvement in this area </li>
<li>Don't use REST/Http for inter-system connect, (Micro-) Service oriented architectures. Point to point performance is horrible. It has its applications in the area of (WAN) web services / platform neutral, easily accessible API's and client/server patterns.</li>
<li>Asynchronous programming is different, requires different/new solution patterns (at source code level). It is unavoidable to learn use of asynchronous messaging primitives. <br />"Pseudo Synchronous" approaches (e.g. fibers) are good in order to better scale multithreading, but do not work out for distributed systems.</li>
<li>lack of craftsmanship can kill visions.</li>
</ul>
<br />
<br />Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com37tag:blogger.com,1999:blog-4356402223581768063.post-88208137293204699822014-12-22T10:56:00.001+01:002015-01-05T02:19:17.707+01:00A persistent KeyValue Server in 40 lines and a sad fact<div>
<em style="background-color: #fcffee; color: #222222; font-family: Verdana, Geneva, sans-serif; font-size: 18px; line-height: 24.6399993896484px;">This post originally was submitted to the <a href="http://www.javaadvent.com/2014/12/a-persistent-keyvalue-server-in-40.html" target="_blank">Java Advent Calendar</a> and is licensed under the <a href="https://creativecommons.org/licenses/by/3.0/" style="color: #888888; text-decoration: none;">Creative Commons 3.0 Attribution</a> license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!</em></div>
<div>
<em style="background-color: #fcffee; color: #222222; font-family: Verdana, Geneva, sans-serif; font-size: 18px; line-height: 24.6399993896484px;"><br /></em>
<em style="background-color: #fcffee; color: #222222; font-family: Verdana, Geneva, sans-serif; font-size: 18px; line-height: 24.6399993896484px;">It also has been published on <a href="https://www.voxxed.com/blog/2014/12/persistent-keyvalue-server-40-lines-sad-fact/" target="_blank">voxxed.com</a> .</em><br />
<em style="background-color: #fcffee; color: #222222; font-family: Verdana, Geneva, sans-serif; font-size: 18px; line-height: 24.6399993896484px;"><br /></em></div>
picking up Peters <a href="http://www.javaadvent.com/2014/12/how-and-why-is-unsafe-used-in-java.html">well written overview</a> on the uses of Unsafe, i'll have a <strike>short</strike> fly-by on how low level techniques in Java can save development effort by enabling a higher level of abstraction <b>or</b> allow for Java performance levels probably unknown to many.<br />
<br />
My major point is to show that conversion of Objects to bytes and vice versa is an important fundamental, affecting virtually any modern java application.<br />
<br />
Hardware enjoys to process streams of bytes, not object graphs connected by pointers as <i>"All memory is tape"<b> </b></i><span style="font-size: x-small; font-weight: bold;">(M.Thompson if I remember correctly ..).</span><br />
<span style="font-size: x-small; font-weight: bold;"><br /></span>Many basic technologies are therefore hard to use with vanilla Java heap objects:<br />
<div>
<ul>
<li><b>Memory Mapped Files</b> - a great and simple technology to persist application data safe, fast & easy.</li>
<li><b>Network communication</b> is based on sending packets of bytes</li>
<li><b>Interprocess communication</b> (shared memory)</li>
<li><b>Large main memory</b> of today's servers (64GB to 256GB). (GC issues)</li>
<li>CPU caches work best on data stored as a continuous stream of bytes in memory</li>
</ul>
</div>
so use of the Unsafe class in most cases boil down in helping to transform a java object graph into a continuous memory region and vice versa either using<br />
<ul>
<li>[performance enhanced] <b>object serialization</b> or</li>
<li><b>wrapper classes</b> to ease access to data stored in a continuous memory region.</li>
</ul>
(Code & examples of this post can be found <a href="https://github.com/RuedigerMoeller/advcalendar2014">here</a>)<br />
<ul></ul>
<span style="font-size: x-large;"><br /></span><b><span style="font-size: x-large;">Serialization based Off-Heap</span></b><br />
<div>
<br />
Consider a retail WebApplication where there might be millions of registered users. We are actually not interested in representing data in a relational database as all needed is a quick retrieve of user related data once he logs in. Additionally one would like to traverse the social graph quickly.<br />
<br />
Let's take a simple user class holding some attributes and a list of 'friends' making up a social graph.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidqoCpjFdqdDtpUOoPlGdzwFcsra2ElbsKlvGRA0d2GVR1PjS_vMkexlkQeD_-Y8j6jFfEv-hVgmQaPJvPxW1erL_2VVmbyWqZrtfDPKOk6h5Uzrb0MzVhIp7w8Roi6LeEL2dYGvgFlQ/s1600/user.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidqoCpjFdqdDtpUOoPlGdzwFcsra2ElbsKlvGRA0d2GVR1PjS_vMkexlkQeD_-Y8j6jFfEv-hVgmQaPJvPxW1erL_2VVmbyWqZrtfDPKOk6h5Uzrb0MzVhIp7w8Roi6LeEL2dYGvgFlQ/s1600/user.png" /></a></div>
<br />
easiest way to store this on heap, is a simple huge HashMap.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Alternatively one can use <b>off heap maps</b> to store large amounts of data. An off heap map stores its keys and values inside the native heap, so garbage collection does not need to track this memory. In addition, native heap can be told to automagically get synchronized to disk (memory mapped files). This even works in case your application crashes, as the OS manages write back of changed memory regions.<br />
<div>
<br /></div>
There are some open source off heap map implementations out there with various feature sets (e.g. <a href="https://github.com/OpenHFT/Chronicle-Map#complex-types">ChronicleMap</a>), for this example I'll use a plain and simple implementation featuring fast iteration (optional full scan search) and ease of use.<br />
<br />
Serialization is used to store objects, deserialization is used in order to pull them to the java heap again. Pleasantly I have written the (afaik) <a href="https://github.com/RuedigerMoeller/fast-serialization">fastest fully JDK compliant object serialization</a> on the planet, so I'll make use of that.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6Zntj14Bc3N4mP6hkm2NllN-RyN1DD5x_U6JdkoCKoIyNFS6R2KvI4duXb0fm0QqfTUCiUiDHu9LuFPsRYjfjIj4DJMgdpwnXb364lqrkDbt4zhU8fd4EMhPPDV1nSOUB14JJI_KwJw/s1600/omap.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6Zntj14Bc3N4mP6hkm2NllN-RyN1DD5x_U6JdkoCKoIyNFS6R2KvI4duXb0fm0QqfTUCiUiDHu9LuFPsRYjfjIj4DJMgdpwnXb364lqrkDbt4zhU8fd4EMhPPDV1nSOUB14JJI_KwJw/s1600/omap.png" /></a></div>
<br />
Done:<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<ul>
<li>persistence by memory mapping a file (map will reload upon creation). </li>
<li>Java Heap still empty to serve real application processing with Full GC < 100ms. </li>
<li>Significantly less overall memory consumption. A user record serialized is ~60 bytes, so in theory 300 million records fit into 180GB of server memory. No need to raise the big data flag and run 4096 hadoop nodes on AWS ;).</li>
</ul>
<div>
<br />
Comparing a regular in-memory java HashMap and a fast-serialization based persistent off heap map holding <b>15 millions</b> user records, will show following results (on a 3Ghz older XEON 2x6):<br />
<br />
<table border="1" cellpadding="0" cellspacing="0" dir="ltr" style="border-collapse: collapse; border: 1px solid rgb(204, 204, 204); font-family: arial, sans, sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="209"></col><col width="165"></col><col width="79"></col><col width="115"></col><col width="115"></col><col width="112"></col></colgroup><tbody>
<tr style="height: 21px;"><td style="background-color: #d9d9d9; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;"></td><td data-sheets-value="[null,2,"consumed Java Heap (MB)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">consumed Java Heap (MB)</td><td data-sheets-value="[null,2,"Full GC (s)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">Full GC (s)</td><td data-sheets-value="[null,2,"Native Heap (MB)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">Native Heap (MB)</td><td data-sheets-value="[null,2,"get/put ops per s"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">get/put ops per s</td><td data-sheets-value="[null,2,"required VM size"]" style="background-color: #d9d9d9; border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">required VM size (MB)</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"HashMap"]" style="background-color: #d9d9d9; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; padding: 2px 3px; vertical-align: bottom;">HashMap</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,6865]" style="padding: 2px 3px; text-align: right; vertical-align: bottom;">6.865,00</td><td data-sheets-numberformat="[null,2,"#,##0.000",1]" data-sheets-value="[null,3,null,26.039]" style="padding: 2px 3px; text-align: right; vertical-align: bottom;">26,039</td><td data-sheets-value="[null,3,null,0]" style="padding: 2px 3px; text-align: right; vertical-align: bottom;">0</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,3800000]" style="padding: 2px 3px; text-align: right; vertical-align: bottom;">3.800.000,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,12000]" style="border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;"><div style="text-align: right;">
12.000,00</div>
</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"OffheapMap (Serialization based)"]" style="background-color: #d9d9d9; border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; padding: 2px 3px; vertical-align: bottom;">OffheapMap (Serialization based)</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,63]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;"><div style="text-align: right;">
63,00</div>
</td><td data-sheets-numberformat="[null,2,"#,##0.000",1]" data-sheets-value="[null,3,null,0.026]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;"><div style="text-align: right;">
0,026</div>
</td><td data-sheets-numberformat="[null,2,"#,###",1]" data-sheets-value="[null,3,null,3050]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;"><div style="text-align: right;">
3.050</div>
</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,750000]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;"><div style="text-align: right;">
750.000,00</div>
</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,500]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;"><div style="text-align: right;">
500,00</div>
</td></tr>
</tbody></table>
<br />
[<a href="https://github.com/RuedigerMoeller/advcalendar2014/blob/master/src/main/java/keyvalue/OffHeapMapExample.java">test source / blog project</a>] Note: You'll need at least 16GB of RAM to execute them.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbfHZka7bokFzrf2J6leJQITpPmhF_xW4nO1CvsNvTNYYO7Mgu5IDtZ7OyKwpm76erM8sgJ-9G_hkbDOCvl57gkN1qJcItzO31HiVgVJ6XS9dS3kDAH3wKEbM3uZ2Sm5Fu9m4_ZM2ySA/s1600/offheap.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbfHZka7bokFzrf2J6leJQITpPmhF_xW4nO1CvsNvTNYYO7Mgu5IDtZ7OyKwpm76erM8sgJ-9G_hkbDOCvl57gkN1qJcItzO31HiVgVJ6XS9dS3kDAH3wKEbM3uZ2Sm5Fu9m4_ZM2ySA/s1600/offheap.jpg" height="133" width="200" /></a></div>
As one can see, even with fast serialization there is a heavy penalty (~factor 5) in access performance, anyway: compared to other persistence alternatives, its still superior (1-3 microseconds per "get" operation, "put()" very similar).<br />
<br />
Use of JDK serialization would perform at least 5 to 10 times slower (direct comparison below) and therefore render this approach useless.<br />
<br />
<br /></div>
<b><span style="font-size: x-large;">Trading performance gains against higher level of abstraction: "Serverize me"</span></b><br />
<span style="font-size: large;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihUWqTB3Xl8mZio2voe98dH5wZvofPl_94ZwBirHcuKujPgndK3PymfH6pcnuMqJZoeLVy8S4HJuJJCAACFmKKv6RZyjtWXsMgitJr4uRb_T37afnAOgRrM1TtsevljCsc_K5-d2oqCA/s1600/parking-like-a-mini-boss-bike.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihUWqTB3Xl8mZio2voe98dH5wZvofPl_94ZwBirHcuKujPgndK3PymfH6pcnuMqJZoeLVy8S4HJuJJCAACFmKKv6RZyjtWXsMgitJr4uRb_T37afnAOgRrM1TtsevljCsc_K5-d2oqCA/s1600/parking-like-a-mini-boss-bike.gif" height="112" width="200" /></a></div>
A single server won't be able to serve (hundreds of) thousands of users, so we somehow need to share data amongst processes, even better: across machines.<br />
<br />
Using a fast implementation, its possible to generously use (fast-) serialization for over-the-network messaging. Again: if this would run like 5 to 10 times slower, it just wouldn't be viable. Alternative approaches require an order of magnitude more work to achieve similar results.<br />
<br />
By wrapping the persistent off heap hash map by an Actor implementation (async ftw!), some lines of code make up a persistent KeyValue server with a TCP-based and a HTTP interface (uses <a href="https://github.com/RuedigerMoeller/kontraktor">kontraktor actors</a>). Of course the Actor can still be used in-process if one decides so later on.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG2E1ptb-SFul0h_pCr1rJft7J1AphbtXriL0DKvmUmY_Sc9-jd5UfP2MejEdHMiRqp6we1yD6NQv_MpI1EkmxFxrwAnSbRFQSpSbpuHwN9J2L_VUIcKBqyZ2_5RHzVvhC_qQAxBieiQ/s1600/kvs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG2E1ptb-SFul0h_pCr1rJft7J1AphbtXriL0DKvmUmY_Sc9-jd5UfP2MejEdHMiRqp6we1yD6NQv_MpI1EkmxFxrwAnSbRFQSpSbpuHwN9J2L_VUIcKBqyZ2_5RHzVvhC_qQAxBieiQ/s1600/kvs.png" height="640" width="604" /></a></div>
<br />
Now that's a micro service. Given it lacks any attempt of optimization and is <b>single threaded</b>, its reasonably fast [same XEON machine as above]:<br />
<ul>
<li>280_000 successful remote lookups per second </li>
<li>800_000 in case of fail lookups (key not found)</li>
<li>serialization based TCP interface (1 liner)</li>
<li>a stringy webservice for the REST-of-us (1 liner).</li>
</ul>
[<a href="https://github.com/RuedigerMoeller/advcalendar2014/tree/master/src/main/java/keyvalue">source: KVServer, KVClient</a>] Note: You'll need at least 16GB of RAM to execute the test.<br />
<br />
A real world implementation might want to double performance by directly putting received serialized object byte[] into the map instead of encoding it twice (encode/decode once for transmission over wire, then decode/encode for offheaping map).<br />
<br />
"RestActorServer.Publish(..);" is a one liner to also expose the KVActor as a webservice in addition to raw tcp:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgZzCMYnPtQxgfyZuJ2biKnL1QWzG0-25F3gPrzdU3fGPS0eO2GY7iL-VZlL4qlsfXbDT76n8G9ieEyEB7cpzY2CVHTstar0DYq_RLlygk72TVnuqtW0UMXDqnsiDL7zC0yNe3h586fw/s1600/Screenshot+from+2014-12-17+00:19:18.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgZzCMYnPtQxgfyZuJ2biKnL1QWzG0-25F3gPrzdU3fGPS0eO2GY7iL-VZlL4qlsfXbDT76n8G9ieEyEB7cpzY2CVHTstar0DYq_RLlygk72TVnuqtW0UMXDqnsiDL7zC0yNe3h586fw/s1600/Screenshot+from+2014-12-17+00:19:18.png" height="304" width="320" /></a></div>
<br />
<br />
<br />
<b><span style="font-size: x-large;">C like performance using flyweight wrappers / structs</span></b><br />
<b><br /></b>With serialization, regular Java Objects are transformed to a byte sequence. One can do the opposite: Create wrapper classes which read data from fixed or computed positions of an underlying byte array or native memory address. (E.g. see <a href="http://mechanical-sympathy.blogspot.de/2012/10/compact-off-heap-structurestuples-in.html">this blog post</a>).<br />
<br />
By moving the base pointer its possible to access different records by just moving the the wrapper's offset. Copying such a "packed object" boils down to a memory copy. In addition, its pretty easy to write allocation free code this way. One downside is, that reading/writing single fields has a performance penalty compared to regular Java Objects. This can be made up for by using the Unsafe class.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgouZX6VpMDX8wiHPl3v7Fdd7y_SMunOWfhjCrZbqcYk_XNvdu2SWiYgIJGrMHLJ3RR6WSgjd-pmydJ44S1ARdGBJcjvXlifdLilmg0Q4APzyUcg__9WXU-GF2JrM2alMNO9ztQMcUqHA/s1600/structs.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgouZX6VpMDX8wiHPl3v7Fdd7y_SMunOWfhjCrZbqcYk_XNvdu2SWiYgIJGrMHLJ3RR6WSgjd-pmydJ44S1ARdGBJcjvXlifdLilmg0Q4APzyUcg__9WXU-GF2JrM2alMNO9ztQMcUqHA/s1600/structs.gif" height="200" width="169" /></a></div>
"flyweight" wrapper classes can be implemented manually as shown in the blog post cited, however as code grows this starts getting unmaintainable.<br />
Fast-serializaton provides a byproduct "struct emulation" supporting creation of flyweight wrapper classes from regular Java classes at runtime. Low level byte fiddling in application code can be avoided for the most part this way.<br />
<br />
<br />
<br />
<div style="text-align: center;">
<i>How a regular Java class can be mapped to flat memory (fst-structs):</i></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMPvzQeoKLpn6YD9f2K96MB_vOHWENsYpVowlnDTIbMtG37pd-iaaXQx6FsUQzXTz4-mL7Lc8QTpVYOoqIeIFQInS_MigyzKIz7pGAF4iTFzyefCJA-vFYRfVZob1RuuJ4T5cUeiORNw/s1600/structs.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg96wjc_zn93hmVzfsyYC1gOBIiUTvBvrAX6YBm2EZa2AD6hxtI4hIQTXbgwNmwi2t1fLkDFI-1gzvQTXU5RsfQHG6Jyb9uMxXZfA5UElDUhJ3CghqZMzksk_jFtJ-u1oC8Cb1Rm-resQ/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg96wjc_zn93hmVzfsyYC1gOBIiUTvBvrAX6YBm2EZa2AD6hxtI4hIQTXbgwNmwi2t1fLkDFI-1gzvQTXU5RsfQHG6Jyb9uMxXZfA5UElDUhJ3CghqZMzksk_jFtJ-u1oC8Cb1Rm-resQ/s1600/Capture.PNG" height="208" width="640" /></a></div>
<br />
Of course there are simpler tools out there to help reduce manual programming of encoding (e.g. <a href="https://github.com/RichardWarburton/slab">Slab</a>) which might be more appropriate for many cases and use less "magic".<br />
<br />
<b>What kind of performance can be expected using the different approaches (sad fact incoming) ?</b><br />
<br />
Lets take the following struct-class consisting of a price update and an embedded struct denoting a tradable instrument (e.g. stock) and encode it using various methods:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbg6gREjjwl6t7T9dh9FheRFL6uLQSArxfCp8pyapE_-A-ll5ElOqBwAXZNdnDVsaHIFNlvEuAzmB07GpCzZd-T5B9M2HAzoxW6J0KEDIrVD9wsZywNQpHAdZ-ZLFQhTx_ZxOkF64-iQ/s1600/struct.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbg6gREjjwl6t7T9dh9FheRFL6uLQSArxfCp8pyapE_-A-ll5ElOqBwAXZNdnDVsaHIFNlvEuAzmB07GpCzZd-T5B9M2HAzoxW6J0KEDIrVD9wsZywNQpHAdZ-ZLFQhTx_ZxOkF64-iQ/s1600/struct.png" height="640" width="532" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i>a 'struct' in code</i></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
<b>Pure encoding performance:</b></div>
<div class="separator" style="clear: both;">
<br /></div>
<table border="1" cellpadding="0" cellspacing="0" dir="ltr" style="border-collapse: collapse; border: 1px solid rgb(204, 204, 204); font-family: arial, sans, sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="100"></col><col width="155"></col><col width="97"></col><col width="130"></col><col width="86"></col></colgroup><tbody>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Structs"]" style="background-color: #d9d9d9; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">Structs</td><td data-sheets-value="[null,2,"fast-Ser (no shared refs)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">fast-Ser (no shared refs)</td><td data-sheets-value="[null,2,"fast-Ser"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">fast-Ser</td><td data-sheets-value="[null,2,"JDK Ser (no shared)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">JDK Ser (no shared)</td><td data-sheets-value="[null,2,"JDK Ser"]" style="background-color: #d9d9d9; border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">JDK Ser</td></tr>
<tr style="height: 21px;"><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,26315000]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">26.315.000,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,7757000]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">7.757.000,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,5102000]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">5.102.000,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,649000]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">649.000,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,644000]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">644.000,00</td></tr>
</tbody></table>
<b><br /></b>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjr_DXP7SzzJ9bk7url-BsCmwcWaU-CzMM2rwW0rsI_wRA_ZrCYSEvIbZNuoR3Kp0-02CENOkWqLWl4cGVwS0AIr0KyfOc6f2ZX_lRGNKYZMO44hUg8KKGG8EiJlDQQDrIgoJ0yC5OrvA/s1600/adv1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjr_DXP7SzzJ9bk7url-BsCmwcWaU-CzMM2rwW0rsI_wRA_ZrCYSEvIbZNuoR3Kp0-02CENOkWqLWl4cGVwS0AIr0KyfOc6f2ZX_lRGNKYZMO44hUg8KKGG8EiJlDQQDrIgoJ0yC5OrvA/s1600/adv1.PNG" /></a></div>
<b><br /></b><b><br /></b><b>Real world test with messaging throughput:</b><br />
<br />
In order to get a basic estimation of differences in a real application, i do an experiment how different encodings perform when used to send and receive messages at a high rate via <a href="https://github.com/RuedigerMoeller/fast-cast">reliable UDP messaging</a>:<br />
<br />
<i>The Test:</i><br />
<i>A sender encodes messages as fast as possible and publishes them using reliable multicast, a subscriber receives and decodes them.</i><br />
<br />
<table border="1" cellpadding="0" cellspacing="0" dir="ltr" style="border-collapse: collapse; border: 1px solid rgb(204, 204, 204); font-family: arial, sans, sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="100"></col><col width="155"></col><col width="97"></col><col width="130"></col><col width="86"></col></colgroup><tbody>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Structs"]" style="background-color: #d9d9d9; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">Structs</td><td data-sheets-value="[null,2,"fast-Ser (no shared refs)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">fast-Ser (no shared refs)</td><td data-sheets-value="[null,2,"fast-Ser"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">fast-Ser</td><td data-sheets-value="[null,2,"JDK Ser (no shared)"]" style="background-color: #d9d9d9; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">JDK Ser (no shared)</td><td data-sheets-value="[null,2,"JDK Ser"]" style="background-color: #d9d9d9; border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(0, 0, 0); border-top-style: solid; border-top-width: 1px; padding: 2px 3px; vertical-align: bottom;">JDK Ser</td></tr>
<tr style="height: 21px;"><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,6644107]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(0, 0, 0); border-left-style: solid; border-left-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">6.644.107,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,4385118]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">4.385.118,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,3615584]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">3.615.584,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,81582]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">81.582,00</td><td data-sheets-numberformat="[null,2,"#,##0.00",1]" data-sheets-value="[null,3,null,79073]" style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(0, 0, 0); border-right-style: solid; border-right-width: 1px; padding: 2px 3px; text-align: right; vertical-align: bottom;">79.073,00</td></tr>
</tbody></table>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp2yKYlcXbOxvoPaaet9wklXU1DtJJ4KnWdfIKfdumrIxEKIJvBRst2D_u6591keT6QWOZYd2KjkzbeSNEZ8_-qbM8GAwktIrfMZMMYY9bmtJY6RY98_1t8A-C_fZXTuV8y6KDvByX-w/s1600/adv2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp2yKYlcXbOxvoPaaet9wklXU1DtJJ4KnWdfIKfdumrIxEKIJvBRst2D_u6591keT6QWOZYd2KjkzbeSNEZ8_-qbM8GAwktIrfMZMMYY9bmtJY6RY98_1t8A-C_fZXTuV8y6KDvByX-w/s1600/adv2.PNG" /></a></div>
(Tests done on I7/Win8, XEON/Linux scores slightly higher, msg size ~70 bytes for structs, ~60 bytes serialization).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsX2iPkoigISuCYfnS1BsB9TIT_8ie2IJjTgRNTq2Tu900-HsaO5ptvcxjzbTfASzKm-9ofUt3jsJeTYKRv26U8w6uLPU0QNNeOlbJXjcr09cVDSRD6Og30d4l-7-yHtdHF73-zKqotg/s1600/omg.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsX2iPkoigISuCYfnS1BsB9TIT_8ie2IJjTgRNTq2Tu900-HsaO5ptvcxjzbTfASzKm-9ofUt3jsJeTYKRv26U8w6uLPU0QNNeOlbJXjcr09cVDSRD6Og30d4l-7-yHtdHF73-zKqotg/s1600/omg.gif" height="120" width="200" /></a></div>
Slowest compared to fastest: factor of 82. The test highlights an issue not covered by micro-benchmarking: Encoding and Decoding should perform similar, as factual throughput is determined by Min(Encoding performance, Decoding performance). For unknown reasons JDK serialization manages to encode the message tested like 500_000 times per second, decoding performance is only 80_000 per second so in the test the receiver gets dropped quickly:<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: Courier New, Courier, monospace;">"</span><br />
<span style="font-family: Courier New, Courier, monospace;">...</span><br />
<span style="font-family: Courier New, Courier, monospace;">***** Stats for receive rate: 80351 per second *********</span><br />
<span style="font-family: Courier New, Courier, monospace;">***** Stats for receive rate: 78769 per second *********</span><br />
<span style="font-family: Courier New, Courier, monospace;">SUB-ud4q has been dropped by PUB-9afs on service 1</span><br />
<span style="font-family: Courier New, Courier, monospace;">fatal, could not keep up. exiting</span><br />
<span style="font-family: Courier New, Courier, monospace;">"</span><br />
(Creating backpressure here probably isn't the right way to address the issue ;-) )<br />
<br />
<b>Conclusion</b>:<br />
<ul>
<li>a fast serialization allows for a level of abstraction in distributed applications impossible if serialization implementation is either<br />- too slow<br />- incomplete. E.g. cannot handle any serializable object graph<br />- requires manual coding/adaptions. (would put many restrictions on actor message types, Futures, Spore's, Maintenance nightmare)</li>
<li>Low Level utilities like Unsafe enable different representations of data resulting in extraordinary throughput or guaranteed latency boundaries (allocation free main path) for particular workloads. These are impossible to achieve by a large margin with JDK's public tool set.</li>
<li>In distributed systems, communication performance is of fundamental importance. Removing Unsafe is not the biggest fish to fry looking at the numbers above .. JSON or XML won't fix this ;-).</li>
<li>While the HotSpot VM has reached an extraordinary level of performance and reliability, CPU is wasted in some parts of the JDK like there's no tomorrow. Given we are living in the age of distributed applications and data, moving stuff over the wire should be easy to achieve (not manually coded) and as fast as possible. </li>
</ul>
<div>
<br /></div>
<div>
<b>Addendum: bounded latency</b></div>
<div>
<br /></div>
A quick <a href="https://github.com/RuedigerMoeller/fast-cast/tree/3.0/examples/src/org/nustaq/fastcast/examples/latency">Ping Pong RTT latency benchmark</a> showing that java can compete with C solutions easily, as long the main path is allocation free and techniques like described above are employed:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5OhpmPm0M00uwkyi5f6YlM08apRv6p_b_BKXnbAeMC7volRbvrukyoq9jHeQrm6xMFOD2l8g3JqswwR3o-gvkVaTdxfQ5qIbo57r7X8OVAQRvIIzSHLqSFQCEFiJZzVOb3p2pmSALAQ/s1600/lat1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5OhpmPm0M00uwkyi5f6YlM08apRv6p_b_BKXnbAeMC7volRbvrukyoq9jHeQrm6xMFOD2l8g3JqswwR3o-gvkVaTdxfQ5qIbo57r7X8OVAQRvIIzSHLqSFQCEFiJZzVOb3p2pmSALAQ/s1600/lat1.png" height="233" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGd_oTGjW211f5LHomufr4mmiJEDufF7Nzk-9SqW7Q4xYkaiG-KhYPq5dOuxa78w4GiaoyPAA_lpsvZdZRYSXUSPMJtHEO9pCNs031tLJTDLt_3-3TYVeMHJL54FuEIZfObmTRaMPiAQ/s1600/lat2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGd_oTGjW211f5LHomufr4mmiJEDufF7Nzk-9SqW7Q4xYkaiG-KhYPq5dOuxa78w4GiaoyPAA_lpsvZdZRYSXUSPMJtHEO9pCNs031tLJTDLt_3-3TYVeMHJL54FuEIZfObmTRaMPiAQ/s1600/lat2.png" height="243" width="640" /></a></div>
<br />
<i>[credits: charts+measurement done with HdrHistogram]</i><br />
<br />
This is an "experiment" rather than a benchmark (so do not read: '<i>Proven: Java faster than C'</i>), it shows low-level-Java can compete with C in at least this low-level domain.<br />
Of course its not exactly <i>idiomatic </i>Java code, however its still easier to handle, port and maintain compared to a JNI or pure C(++) solution. Low latency C(++) code won't be that idiomatic either ;-)<br />
<br />
<b>About me:</b> I am a solution architect freelancing at an exchange company in the area of realtime GUIs, middleware, and low latency CEP (Complex Event Processing) nightly hacking at <a href="https://github.com/RuedigerMoeller">https://github.com/RuedigerMoeller</a>.<br />
<br />
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com441tag:blogger.com,1999:blog-4356402223581768063.post-1884416616828291432014-10-28T19:59:00.003+01:002014-10-30T12:27:32.343+01:00The Internet is running in debug modeWith the rise of the Web, textual encodings like xml/JSON have become very popular. Of course textual message encoding has many advantages from a developer's perspective. What most developers are not aware of, is how expensive encoding and decoding of textual messages really is compared to a well defined binary protocol.<br />
<br />
Its common to define a system's behaviour by its <b>protocol</b>. Actually, a protocol messes up two distinct aspects of communication, namely:<br />
<br />
<ul>
<li>Encoding of messages</li>
<li>Semantics and behavior (request/response, signals, state transition of communication parties ..)</li>
</ul>
<br />
Frequently (not always), these two very distinct aspects are mixed up without need. So we are forced to run the whole internet in "<b>debug mode</b>", as 99% of webservice and webapp communication is done using textual protocols.<br />
<br />
The overhead in CPU consumption compared to a well defined binary encoding is factor ~3-5 (JSON) up to >10-20 (XML). The unnecessary waste of bandwith also adds to that greatly (yes you can zip, but this in turn will waste even more CPU).<br />
<br />
I haven't calculated the numbers, but this is <b>environmental pollution at big scale. </b>Unnecessary CPU consumption to this extent wastes <b>a lot</b> of energy (global warming anyone ?).<br />
<br />
Solution is easy:<br />
<ul>
<li>Standardize on some simple encodings (pure binary, self describing binary ("binary json"), textual)</li>
<li>Define the behavioral part of a protocol (exclude encoding)</li>
<li>use textual encoding during development, use binary in production.</li>
</ul>
Man we could save a lot of webserver hardware if only http headers could be binary encoded ..Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com136tag:blogger.com,1999:blog-4356402223581768063.post-1403115301617230982014-10-17T22:10:00.000+02:002014-10-18T01:02:29.336+02:00Follow up: Executors and Cache Locality ExperimentThanks to <a href="http://jpbempel.blogspot.de/" target="_blank">Jean Philippe Bempel</a> who challenged my results (for a reason), I discovered an issue in last post: Code-completion let me accidentally choose Executors.newSingleThread<b>Scheduled</b>Executor() instead of Executors.newSingleThreadExecutor(), so the pinned-to-thread-actor results are actually even better than reported previously. The big picture has not changed that much, but its still worthwhile reporting.<br />
<br />
On a second note: There are many other aspects to concurrent scheduling such as queue implementations etc.. Especially if there is no "beef" inside the message processing, these differences become more dominant compared to cache misses, but this is another problem that has been covered extensively by other people in depth (e.g. <a href="http://psy-lob-saw.blogspot.de/" target="_blank">Nitsan Wakart</a>).<br />
<br />
Focus of this experiment is locality/cache misses, keep in mind different queueing implementations of executors for sure add dirt/bias.<br />
<br />
As requested, I add results from the linux "perf" tool to prove there are significant differences in cache misses caused by random assignment of Thread - to Actor as done by ThreadPoolExecutor and WorkStealingExecutor.<br />
<br />
Check out my <a href="http://java-is-the-new-c.blogspot.de/2014/10/experiment-cache-effects-when.html" target="_blank">recent post</a> for a description of the test case.<br />
<br />
<span style="font-size: large;"><b>Results with adjusted SingleThreadExecutor (XEON 2 socket, each 6 cores, no HT)</b></span><br />
<span style="font-size: large;"><b><br /></b></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl7Wed9O7E1l98G6MCrX7zDhWkLX3BA1Z71QniPW0wwpFRU5MbhWcJQFBnPAGoRYn1uKc9rPQnUTjVge-e2pKJMBnW6i-yjL4WTQ-zMpZppRqvGZqnQlJtshunJmrj9V55Q4oOx24gIEg/s1600/2cnd-8-80-100.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl7Wed9O7E1l98G6MCrX7zDhWkLX3BA1Z71QniPW0wwpFRU5MbhWcJQFBnPAGoRYn1uKc9rPQnUTjVge-e2pKJMBnW6i-yjL4WTQ-zMpZppRqvGZqnQlJtshunJmrj9V55Q4oOx24gIEg/s1600/2cnd-8-80-100.png" height="238" width="640" /></a></div>
As in previous post, "dedicated" actor-pinned-to-thread performs best. For very small local state, there are only few cache misses so differences are small, but widen once a bigger chunk of memory is accessed by each actor. Note that ThreadPool is hampered by its internal scheduling/queuing mechanics, regardless of locality, it performs weak.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__SUCxqNc7if-LrtxhvhLVS-_ZAtiogdbvRzKKzCaZCro9zyJErg-drqmXMXPmyitkv11PVFTI7DY_4t3Dg8xL6KAdAvMkKUCqbTPjwJlKzYaafXyjqOlbvf3he3072pHgOTPAGVYPgM/s1600/2cnd-8-8000-100.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__SUCxqNc7if-LrtxhvhLVS-_ZAtiogdbvRzKKzCaZCro9zyJErg-drqmXMXPmyitkv11PVFTI7DY_4t3Dg8xL6KAdAvMkKUCqbTPjwJlKzYaafXyjqOlbvf3he3072pHgOTPAGVYPgM/s1600/2cnd-8-8000-100.png" height="237" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
When increasing number of Actors to 8000 (so 1000 actors per thread), "Workstealing" and "Dedicated" perform similar. Reason: executing 8000 actors round robin creates cache misses for both executors. Note that in a real world server its likely that there are active and inactive actors, so I'd expect "Dedicated" to perform slightly better than in this synthetic test.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b><span style="font-size: large;">"perf stat -e" and "perf stat -cs" results</span></b></div>
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<div style="clear: both; text-align: left;">
(only<span style="font-family: inherit;"> <span style="background-color: white; color: #222222; font-size: 13px;">2000, 4000, 8000 local size tests where run)</span></span></div>
<div style="clear: both; text-align: left;">
<span style="font-family: inherit;"><span style="background-color: white; color: #222222; font-size: 13px;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: left;">
<b><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">Dedicated/actor-pinned-to-</span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">thread:</span></b></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><b>333,669,424</b> cache-misses </span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"> </span></div>
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">19.996366007 seconds time elapsed</span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">185,440 context-switches </span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"> </span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">20.230098005 seconds time elapsed</span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">=> <b>9,300 context switches per second</b></span><br />
<br style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;" />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><b>workstealing:</b></span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><b>2,524,777,488</b> cache-misses </span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"> </span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">39.610565607 seconds time elapsed</span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">381,385 context-switches </span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"> </span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">39.831169694 seconds time elapsed</span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">=> <b>9,500 </b></span><b style="color: #222222; font-family: arial, sans-serif; font-size: 12.7272720336914px;">context switches </b><b style="color: #222222; font-family: arial, sans-serif; font-size: 13px;">per second</b><br />
<div class="" style="clear: both; text-align: left;">
<br style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;" />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><b>fixedthreadpool:</b></span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><b>3,213,889,492</b> cache-misses </span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"> </span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">92.141264115 seconds time elapsed</span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">25,387,972 context-switches </span><wbr style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"> </span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">87.547306379 seconds time elapsed</span><br />
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">=><b>290,000 context switches per second</b></span></div>
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii7ZijK2llaOc47Ifbhn9N7hLvIrpxCu-K5fz0MQmxinSVMFVdmGo3drl52pPmZvjMsL0bW_apTAZysoYdNtfkjN4JpEFEdsYu45V-v97dROuZ49-nvold76Hss52MMU-cOJj7LU3XECU/s1600/perf-context-switch.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQkgaP68oNJg9dUPpNB6Ri_U4XGgX4XvoPU0zjV-ituE6cyKYofytkvNmESheApia4AYlW_eQVCGzHZTrVonmN3Vjn9JmzDl0_bXLQeFnszjuUmTuCdrDWDpNF34cWbej3JCbHGpGgnus/s1600/perf-misses.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii7ZijK2llaOc47Ifbhn9N7hLvIrpxCu-K5fz0MQmxinSVMFVdmGo3drl52pPmZvjMsL0bW_apTAZysoYdNtfkjN4JpEFEdsYu45V-v97dROuZ49-nvold76Hss52MMU-cOJj7LU3XECU/s1600/perf-context-switch.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii7ZijK2llaOc47Ifbhn9N7hLvIrpxCu-K5fz0MQmxinSVMFVdmGo3drl52pPmZvjMsL0bW_apTAZysoYdNtfkjN4JpEFEdsYu45V-v97dROuZ49-nvold76Hss52MMU-cOJj7LU3XECU/s1600/perf-context-switch.png" height="223" width="400" /></a><strike><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQkgaP68oNJg9dUPpNB6Ri_U4XGgX4XvoPU0zjV-ituE6cyKYofytkvNmESheApia4AYlW_eQVCGzHZTrVonmN3Vjn9JmzDl0_bXLQeFnszjuUmTuCdrDWDpNF34cWbej3JCbHGpGgnus/s1600/perf-misses.png" height="216" width="400" /></strike><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQkgaP68oNJg9dUPpNB6Ri_U4XGgX4XvoPU0zjV-ituE6cyKYofytkvNmESheApia4AYlW_eQVCGzHZTrVonmN3Vjn9JmzDl0_bXLQeFnszjuUmTuCdrDWDpNF34cWbej3JCbHGpGgnus/s1600/perf-misses.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a></div>
<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;"><b>A quick test with a more realistic test method</b></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;"><b><br /></b></span></div>
<div class="separator" style="clear: both; text-align: left;">
In order to get a more realistic impression I replaced the synthetic int-iteration by some dirty "real world" dummy stuff (do some allocation and HashMap put/get). Instead of increasing the size of the "localstate" int array, I increase the HashMap size (should also have negative impact on locality).</div>
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxyIt_3VnMDq2b9cqBQAQc75T7G_QXix_Svfy1f8ehcW4waT-Pm-3_bqouhevJpM350y_lYHx5ofkS6FVFUWHO0tD6obTPjrSOZzPhirVb2-t6npPezbhgdG6qGUhMqfcKOw5eGaJ-RIE/s1600/method.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxyIt_3VnMDq2b9cqBQAQc75T7G_QXix_Svfy1f8ehcW4waT-Pm-3_bqouhevJpM350y_lYHx5ofkS6FVFUWHO0tD6obTPjrSOZzPhirVb2-t6npPezbhgdG6qGUhMqfcKOw5eGaJ-RIE/s1600/method.png" height="290" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: left;">
Note that this is rather short processing, so queue implementations and executor internal implementation might dominate locality here. This test is run on Opteron 8c16t * 2Sockets, a processor with 8kb L1 cache size only. (BTW: impl is extra dirty, so no performance optimization comments pls, thx)</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5VtnO4GT-AALliL27-GTElEq4in6-kJBPLcfLBKvG6fsTqIOzu5fzMooqBSC88spPN8VgrPG_GK_9q6q_290WcsWpEA2zopREH_wFPxfBSPRYle1qXhXfR9GghLcJHFOMIEv0BAjk2Ow/s1600/2cnd-real-load.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5VtnO4GT-AALliL27-GTElEq4in6-kJBPLcfLBKvG6fsTqIOzu5fzMooqBSC88spPN8VgrPG_GK_9q6q_290WcsWpEA2zopREH_wFPxfBSPRYle1qXhXfR9GghLcJHFOMIEv0BAjk2Ow/s1600/2cnd-real-load.png" height="230" width="640" /></a></div>
<br />
As ThreadPoolExecutor is abnormous bad in this Test/Processor combination, plain numbers:<br />
<br />
<table border="1" cellpadding="0" cellspacing="0" dir="ltr" style="border-collapse: collapse; border: 1px solid #ccc; font-family: arial,sans,sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="160"></col><col width="172"></col><col width="167"></col><col width="203"></col><col width="165"></col><col width="100"></col><col width="100"></col></colgroup><tbody>
<tr style="height: 21px;"><td style="padding: 2px 3px 2px 3px; vertical-align: bottom;"></td><td data-sheets-value="[null,2,"64 entries"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">64 HMap entries</td><td data-sheets-value="[null,2,"256 entries"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">256 <span style="font-size: 12.7272720336914px;">HMap</span><span style="font-size: 100%;">entries</span></td><td data-sheets-value="[null,2,"2000 entries"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">2000 <span style="font-size: 12.7272720336914px;">HMap</span><span style="font-size: 100%;">entries</span></td><td data-sheets-value="[null,2,"4000 entries"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">4000 <span style="font-size: 12.7272720336914px;">HMap</span><span style="font-size: 100%;">entries</span></td><td data-sheets-value="[null,2,"32k entries"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">32k <span style="font-size: 12.7272720336914px;">HMap</span><span style="font-size: 100%;">entries</span></td><td data-sheets-value="[null,2,"320k entries"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">320k <span style="font-size: 12.7272720336914px;">HMap</span><span style="font-size: 100%;">entries</span></td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"WorkStealing"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">WorkStealing</td><td data-sheets-value="[null,3,null,1070]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1070</td><td data-sheets-value="[null,3,null,1071]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1071</td><td data-sheets-value="[null,3,null,1097]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1097</td><td data-sheets-value="[null,3,null,1129]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1129</td><td data-sheets-value="[null,3,null,1238]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1238</td><td data-sheets-value="[null,3,null,1284]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">1284</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"Dedicated"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">Dedicated</td><td data-sheets-value="[null,3,null,656]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">656</td><td data-sheets-value="[null,3,null,646]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">646</td><td data-sheets-value="[null,3,null,661]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">661</td><td data-sheets-value="[null,3,null,649]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">649</td><td data-sheets-value="[null,3,null,721]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">721</td><td data-sheets-value="[null,3,null,798]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">798</td></tr>
<tr style="height: 21px;"><td data-sheets-value="[null,2,"ThreadPool"]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; vertical-align: bottom;">ThreadPool</td><td data-sheets-value="[null,3,null,8314]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">8314</td><td data-sheets-value="[null,3,null,8751]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">8751</td><td data-sheets-value="[null,3,null,9412]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">9412</td><td data-sheets-value="[null,3,null,9492]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">9492</td><td data-sheets-value="[null,3,null,10269]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">10269</td><td data-sheets-value="[null,3,null,10602]" style="background-color: white; color: #222222; font-size: 100%; padding: 2px 3px 2px 3px; text-align: right; vertical-align: bottom;">10602</td></tr>
</tbody></table>
<br />
Conclusions basically stay same as in original post. Remember cache misses are only one factor of overall runtime performance, so there are workloads where results might look different. Quality/specialization of queue implementation will have huge impact in case processing consists of only some lines of code.<br />
<br />
Finally, my result:<b> </b><br />
<blockquote class="tr_bq">
<b><i>Pinning actors to threads created lowest cache miss rates in any case tested.</i></b></blockquote>
<br />
<span style="font-size: large;"><b><br /></b></span>
<br />
<br />
<!-- Blogger automated replacement: "https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?url=http%3A%2F%2F2.bp.blogspot.com%2F-2lJPvYVD8C8%2FVEFtG_b_5QI%2FAAAAAAAAAXM%2FUcD9myFuh4M%2Fs1600%2Fperf-context-switch.png&container=blogger&gadget=a&rewriteMime=image%2F*" with "https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii7ZijK2llaOc47Ifbhn9N7hLvIrpxCu-K5fz0MQmxinSVMFVdmGo3drl52pPmZvjMsL0bW_apTAZysoYdNtfkjN4JpEFEdsYu45V-v97dROuZ49-nvold76Hss52MMU-cOJj7LU3XECU/s1600/perf-context-switch.png" -->Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com100tag:blogger.com,1999:blog-4356402223581768063.post-62740798078557090622014-10-14T21:57:00.002+02:002014-10-17T22:25:59.732+02:00Experiment: Cache effects when scheduling Actors with F/J, Threadpool, Dedicated Threads<b>Update: I accidentally used newSingleThreadScheduledExecutor instead of newFixedThreadPool(1)</b> <b>for the "Dedicated" test case [ide code completion ..].</b> <b>With this corrected, "Dedicated" outperforms even more.</b> <b>See <a href="http://java-is-the-new-c.blogspot.de/2014/10/follow-up-executors-and-cache-locality.html" target="_blank">follow up post</a> for updated results + "perf" tool cache miss measurement results (do not really change the big picture).</b><br />
<br />
The experiment in my last post had a <span style="color: black;"><b>serious flaw</b></span>: In an actor system, operations on a single actor are executed one after the other. However by naively adding message-processing jobs to executors, private actor state was accessed concurrently, leading to "false-sharing" and cache coherency related costs especially for small local state sizes.<br />
<br />
Therefore I modified the test. For each Actor scheduled, the next message-processing is scheduled once the previous one finished, so the experiment resembles the behaviour of typical actors (or lightweight processes/tasks/fibers) correctly without concurrent access to a memory region.<br />
<br />
Experiment roundup:<br />
<br />
Several million messages are scheduled to several "Actor" simulating classes. Message processing is simulated by reading and writing the private, actor-local state in random order. There are more Actors (24-8000) than threads (6-8). Note that results established (if any) will also hold true for other light-weight concurrency schemes like go-routines, fibers, tasks ...<br />
<br />
The test is done with<br />
<br />
<ul>
<li>ThreadPoolExecutor</li>
<li>WorkStealingExecutor</li>
<li>Dedicated Thread (Each Actor has a fixed assignment to a worker thread)</li>
</ul>
<br />
Simulating an Actor accessing local state:<br />
<ul>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvGMxXt2tHdt2d7KBuhow4xtNX7zI_jsgVL-G2BADVpjqKPB6DFexU0cIb460J0c7wYqLz4rm2iBImza6Dj-mgaPZDcyVbxEQ-uTXGUsIdbq8CadwHs7MHN0bFlha5ElLTQPMM7P-c93M/s1600/work2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvGMxXt2tHdt2d7KBuhow4xtNX7zI_jsgVL-G2BADVpjqKPB6DFexU0cIb460J0c7wYqLz4rm2iBImza6Dj-mgaPZDcyVbxEQ-uTXGUsIdbq8CadwHs7MHN0bFlha5ElLTQPMM7P-c93M/s1600/work2.png" /></a></div>
<b><br /></b>
<b></b><br />
<a href="https://github.com/RuedigerMoeller/executortest/tree/master/src/main/java/exec2" target="_blank">Full Source of Benchmark</a><br />
<br />
<b>Suspection</b>:<br />
<blockquote class="tr_bq">
<b><i>As ThreadPoolExecutor and WorkStealingExecutor schedule each message on a random Thread, they will produce more cache misses compared to pinning each actor onto a fixed thread. Speculation is, that work stealing cannot make up for the costs provoked by cache misses.</i></b></blockquote>
<b><br /></b>
<b><br /></b>
<b>(Some) Variables:</b><br />
<ul>
<li>Number of worker threads</li>
<li>Number of actors</li>
<li>Amount of work per message</li>
<li>Locality / Size of private unshared actor state</li>
</ul>
<br />
<br />
<b><span style="font-size: large;">8 Threads 24 actors 100 memory accesses </span></b><b><span style="font-size: large;">(per msg)</span></b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinbTq598zWJSJsp1KT-P8Upy8wsIUBAyNnPeh3y95YPOxRgUqJMJWIZ2bRqyw380EbUnivAiHhTm_oIV_pMz1VtrRxHEFRQ1zFg0nDLQa9fVbdfruYyPmMJT_ex0RBDdurZ1dYDnj71v8/s1600/8-24-100.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinbTq598zWJSJsp1KT-P8Upy8wsIUBAyNnPeh3y95YPOxRgUqJMJWIZ2bRqyw380EbUnivAiHhTm_oIV_pMz1VtrRxHEFRQ1zFg0nDLQa9fVbdfruYyPmMJT_ex0RBDdurZ1dYDnj71v8/s1600/8-24-100.png" height="242" width="640" /></a></div>
<br />
<b>Interpretation:</b><br />
<b><br /></b>
For this particular load, fixed assigned threads outperform executors. Note: the larger the local state of an actor, the higher the probability of a prefetch fail => cache miss. In this scenario my suspection holds true: Work stealing cannot make up for the amount of cache misses. fixed assigned threads profit, because its likely, some state of a previously processed message resides still in cache once a new message is processed on an actor.<br />
Its remarkable how bad ThreadpoolExecutor performs in this experiment.<br />
<br />
This is a scenario typical for backend-type service: There are few actors with high load. When running a front end server with many clients, there are probably more actors, as typically there is one actor per client session. Therefor lets push up the number of actors to 8000:<br />
<br />
<b><span style="font-size: large;">8 Threads 8000 actors 100 memory accesses (per msg)</span></b><br />
<b style="font-size: x-large;"><br /></b>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiiaT4sDxIxRFmxIzVh0uQVoBk2kc7OhnGMYqIcVYR6jCulCbReT3bKdv_3gO6Gtm9VDTiXSyDBUSiUlTum3J4WetFRVDZuKTH6Py9gbRUa2l5xCW0iUca7GxUdq1KBGTnuXFZ1ELQdDA/s1600/8-8000-100.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiiaT4sDxIxRFmxIzVh0uQVoBk2kc7OhnGMYqIcVYR6jCulCbReT3bKdv_3gO6Gtm9VDTiXSyDBUSiUlTum3J4WetFRVDZuKTH6Py9gbRUa2l5xCW0iUca7GxUdq1KBGTnuXFZ1ELQdDA/s1600/8-8000-100.png" height="234" width="640" /></a></div>
<b style="font-size: x-large;"><br /></b>
<b>Interpretation:</b><br />
<div>
<b><br /></b></div>
<div>
With this amount of actors, all execution schemes suffer from cache misses, as the accumulated size of 8000 actors is too big to fit into L1 cache. Therefore the cache advantage of fixed-assigned threads ('Dedicated') does not make up for the lack of work stealing. Work Stealing Executor outperforms any other execution scheme if a large amount of state is involved.</div>
<div>
This is a somewhat unrealistic scenario as in a real server application, client request probably do not arrive "round robin", but some clients are more active than others. So in practice I'd expect "Dedicated" will at least have some advantage of higher cache hits. Anyway: when serving many clients (stateful), WorkStealing could be expected to outperform.</div>
<div>
<br /></div>
<div>
Just to get a third variant: same test with 240 actors:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSEqB6isoUtBfm94m9BNM_7JZjaT9p8TC9E6Wlm4Nzd4tuibxGG3j_2faiuYT_2HCMLAbpN-B2DW04-8hmZ0fE-R1iZ_4Wstnv_42Er8999A2uf9f35h1kwfn4kZpT3TwUicIXATZlVnQ/s1600/8-240-100.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSEqB6isoUtBfm94m9BNM_7JZjaT9p8TC9E6Wlm4Nzd4tuibxGG3j_2faiuYT_2HCMLAbpN-B2DW04-8hmZ0fE-R1iZ_4Wstnv_42Er8999A2uf9f35h1kwfn4kZpT3TwUicIXATZlVnQ/s1600/8-240-100.png" height="238" width="640" /></a></div>
<div>
<br /></div>
<div>
These results complete the picture: with fewer actors, cache effect supercede work stealing. The higher the number of actors, the higher the number of cache misses gets, so work stealing starts outperforming dedicated threads.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b style="font-size: x-large;">Modifying other variables</b></div>
<div>
<b style="font-size: x-large;"><br /></b></div>
<div>
<b>Number of memory accesses</b></div>
<div>
<b><br /></b></div>
<div>
If a message-processing does few memory accesses, work stealing improves compared to the other 2. Reason: fewer memory access means fewer cache misses means work stealing gets more significant in the overall result.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> ************** Worker Threads:8 actors:24 #mem accesses: 20</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 64 WorkStealing avg:505</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 64 ThreadPool avg:2001</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 64 Dedicated avg:557</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 256 WorkStealing avg:471</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 256 </span><span style="background-color: white; color: #222222; font-size: 12.7272720336914px;">ThreadPool</span><span style="background-color: white; color: #222222; font-size: 13px;"> avg:1996</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 256 Dedicated avg:561</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 2000 WorkStealing avg:589</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 2000 </span><span style="background-color: white; color: #222222; font-size: 12.7272720336914px;">ThreadPool</span><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2109</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 2000 Dedicated avg:600</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 4000 WorkStealing avg:625</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 4000 </span><span style="background-color: white; color: #222222; font-size: 12.7272720336914px;">ThreadPool</span><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2096</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 4000 Dedicated avg:600</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 32000 WorkStealing avg:687</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 32000 </span><span style="background-color: white; color: #222222; font-size: 12.7272720336914px;">ThreadPool</span><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2328</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 32000 Dedicated avg:640</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 320000 WorkStealing avg:667</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 320000 </span><span style="background-color: white; color: #222222; font-size: 12.7272720336914px;">ThreadPool</span><span style="background-color: white; color: #222222; font-size: 13px;"> avg:3070</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 320000 Dedicated avg:738</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 3200000 WorkStealing avg:1341</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 3200000 </span><span style="background-color: white; color: #222222; font-size: 12.7272720336914px;">ThreadPool</span><span style="background-color: white; color: #222222; font-size: 13px;"> avg:3997</span></span></div>
<div>
<span style="background-color: white; color: #222222; font-size: 13px;"><span style="font-family: Courier New, Courier, monospace;">local state bytes: 3200000 Dedicated avg:1428</span></span></div>
<div>
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><br /></span></div>
<div>
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><br /></span></div>
<div>
<b>Fewer worker threads</b></div>
<div>
<b><br /></b></div>
<div>
Fewer worker threads (e.g. 6) increase probability of an actor message being scheduled to the "right" thread "by accident", so cache miss penalty is lower which lets work stealing perform better than "Dedicated" (the fewer threads used, the lower the cache advantage of fixed assigned "Dedicated" threads). Vice versa: if the number of cores involved increases, fixed thread assignment gets ahead.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;">Worker Threads:6 actors:18 #mem accesses: 100</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 64 WorkStealing avg:2073</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 64 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2498</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 64 Dedicated avg:2045</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 256 WorkStealing avg:1735</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 256 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2272</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 256 Dedicated avg:1815</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 2000 WorkStealing avg:2052</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 2000 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2412</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 2000 Dedicated avg:2048</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 4000 WorkStealing avg:2183</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 4000 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2373</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 4000 Dedicated avg:2130</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 32000 WorkStealing avg:3501</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 32000 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:3204</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 32000 Dedicated avg:2822</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 320000 WorkStealing avg:3089</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 320000 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:2999</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 320000 Dedicated avg:2543</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 3200000 WorkStealing avg:6579</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 3200000 </span></span><span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 12.7272720336914px;">ThreadPool</span><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: white; color: #222222; font-size: 13px;"> avg:6047</span><br style="background-color: white; color: #222222; font-size: 13px;" /><span style="background-color: white; color: #222222; font-size: 13px;">local state bytes: 3200000 Dedicated avg:6907</span></span></div>
<div>
<br /></div>
<div>
Machine tested:</div>
<div>
<br /></div>
<div>
(real cores no HT)</div>
<div>
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">$ lscpu </span></div>
<div>
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Architecture: x86_64</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">CPU op-mode(s): 32-bit, 64-bit</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Byte Order: Little Endian</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">CPU(s): 12</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">On-line CPU(s) list: 0-11</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Thread(s) per core: 1</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Core(s) per socket: 6</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Socket(s): 2</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">NUMA node(s): 2</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Vendor ID: GenuineIntel</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">CPU family: 6</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Model: 44</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Stepping: 2</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">CPU MHz: 3067.058</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">BogoMIPS: 6133.20</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">Virtualization: VT-x</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">L1d cache: 32K</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">L1i cache: 32K</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">L2 cache: 256K</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">L3 cache: 12288K</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">NUMA node0 CPU(s): 1,3,5,7,9,11</span><br />
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;">NUMA node1 CPU(s): 0,2,4,6,8,10</span></div>
<div>
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;"><br /></span></div>
<div>
<span style="background-color: white; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 13px;"><br /></span></div>
<div>
<b><span style="font-size: x-large;">Conclusion</span></b></div>
<div>
<ul>
<li><span style="font-family: inherit;">Performance of executors depends heavy on use case. There are work loads where cache locality dominates, giving an advantage of up to 30% over Work-Stealing Executor</span></li>
<li><span style="font-family: inherit;">Performance of executors varies amongst different CPU types and models (L1 cache size + cost of a cache miss matter here)</span></li>
<li><span style="font-family: inherit;">WorkStealing could be viewed as the better overall solution. Especially if a lot of L1 cache misses are to be expected anyway.</span></li>
<li><span style="font-family: inherit;">The <b>ideal executor would be WorkStealing with a soft actor-to-thread affinitiy</b>. This would combine the strength of both execution schemes and would yield significant performance improvements for many workloads</span></li>
<li><span style="font-family: inherit;">Vanilla thread pools without work stealing and actor-to-thread affinity perform significantly worse and should not be used to execute lightweight processes.</span></li>
</ul>
<a href="https://github.com/RuedigerMoeller/executortest/tree/master/src/main/java/exec2" target="_blank">Source of Benchmark</a></div>
<div>
<br style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;" /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com32tag:blogger.com,1999:blog-4356402223581768063.post-3383020716718658542014-10-12T18:03:00.000+02:002014-10-14T22:05:26.174+02:00Alternatives to Executors when scheduling Tasks/Actors<br />
<br />
Executors work well when fed with short units of stateless work, e.g.: division of a computation onto many CPU's. However they are sub-optimal to schedule jobs which are part of an ongoing, larger unit of work, e.g.: Scheduling messages of an actor or lightweight process.<br />
<br />
Many Actor Frameworks (or similar message passing concurrency frameworks) schedule batches of messages using an Executor service. As the Executor service is not context aware, this spreads out processing of a single Actor/Task's messages across several threads/CPU's.<br />
<br />
This can lead to many cache misses when accessing the state of an Actor/Process/Task as the processing thread changes frequently.<br />
Even worse, this way CPU caches cannot stabilize as each new "Runnable" washes out cached memory of the previously processed task's.<br />
<br />
A second issue arises when using busy-spin polling. If a framework reads its queues using busy-spin, it generates 100% CPU load for each processing Thread. So one would like to add a second thread/core only if absolutely required, so a one-thread-per-actor policy is not feasible.<br />
<br />
With Kontraktor 2.0 I implemented a different scheduling mechanism, which achieves a horizontal scaling using a very simple metric to measure actual application CPU requirements, without randomly spreading processing of an actor onto different Cores's.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoxzjAzdJU2FIblus9kQA4EGQAbJqhycWVrTIHykN5vGeA-83ep-mailkHP7ovP5XLqxr7d21JjdKw-Z65L2rNaFRmrz3T2kmmAhpjYVICX5bxN6h3hMVOV92jmhvmtElvS9z50XRoEDM/s1600/actor+scheduling.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoxzjAzdJU2FIblus9kQA4EGQAbJqhycWVrTIHykN5vGeA-83ep-mailkHP7ovP5XLqxr7d21JjdKw-Z65L2rNaFRmrz3T2kmmAhpjYVICX5bxN6h3hMVOV92jmhvmtElvS9z50XRoEDM/s1600/actor+scheduling.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
An actor has a fixed assignment to a workerthread ("DispatcherThread"). The scheduler periodically reschedules Actors by moving them to another worker if necessary.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Since overly sophisticated algorithms tend to introduce high runtime costs, actual scheduling is done in a very simple manner:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
1) a Thread is observed as being overloaded, if the consume loop (pulling messages from the actor queue) has not been idle for N messages (currently N=1000).</div>
<div class="separator" style="clear: both; text-align: left;">
2) If a Thread has been marked "overloaded", Actors with largest mailbox sizes are migrated to a newly started Thread (until #Threads == ThreadMax) as long SUM_QUEUED_MSG(Actors scheduled on Thread A) > SUM_QUEUED_MSG(Actors scheduled on newly created Thread B).</div>
<div class="separator" style="clear: both; text-align: left;">
3) in case #Threads == ThreadMax, actors are rebalanced based on summation of queued messages and "overload" observations.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Problem areas: </div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li>If the processing time of messages vary a lot, summation of queued messages is misleading. An improvement would be to add a weight onto each message type by profiling periodically. A simpler option would be to give an Actor an additional weight to be multiplicated with its queue size.</li>
<li>For load bursts, there is a latency until all of the available CPU's are actually used.</li>
<li>The delay until JIT kicks in produces bad profiling data and leads to false scale ups (heals over time, so not that bad).</li>
</ul>
<div>
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;">An experiment</span></b><br />
<b><span style="font-size: large;"><br /></span><span style="color: #660000;"><span style="background-color: #990000;"><span style="background-color: white;">Update: Woke up this morning and it came to my mind this experiment has a flaw, as jobs per workitem are scheduled in parallel for the executor tests, so I am probably measuring the effects of false sharing. Results below removed, check the <a href="http://java-is-the-new-c.blogspot.de/2014/10/experiment-cache-effects-when.html" target="_blank">follow up post.</a></span></span></span></b><br />
<br />
<br />
</div>
<div>
<b><span style="font-size: large;">Performance cost of adaptive Scaling</span></b></div>
<br />
<br />
To measure cost of auto-scheduling vs. explicit Actor=>Thread assignment, I run the Computing-Pi Test (see previous posts).<br />
<b>These numbers do not show effects of locality as it compares explicit scheduling with automatic scheduling</b>.<br />
<br />
<b>Test 1</b> manually assigns a Thread to each Pi computing Actor,<br />
<b>Test 2</b> always starts with one worker and needs to scale up automatically once it detects actual load.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivitvmnMHNYRazr8yOJBkbwZDS_xat8qMnk3jiIeYxCe76QQjB1wtEqOVM55rE3r478VlT9TcGSw1FR7ZU0i8kwhmWJY3TjCsRf8wac9NAL0jY96wi2fdzwldQb-e-xuAZzfiwjD2Waww/s1600/kpi.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivitvmnMHNYRazr8yOJBkbwZDS_xat8qMnk3jiIeYxCe76QQjB1wtEqOVM55rE3r478VlT9TcGSw1FR7ZU0i8kwhmWJY3TjCsRf8wac9NAL0jY96wi2fdzwldQb-e-xuAZzfiwjD2Waww/s1600/kpi.png" /></a></div>
<br />
(note example requires >= kontraktor2.0-beta-2, if the 'parkNanos' stuff is uncommented it scales up to 2-3 threads only)<br />
<br />
The test is run 8 times with increasing <b>thread_max</b> by one with each run.<br />
<br />
results:<br />
<br />
Kontraktor Autoscale (always start with 1 thread, then scale up to N threads)<br />
<br />
1 threads : 1527<br />
2 threads : 1273<br />
3 threads : 718<br />
4 threads : 630<br />
5 threads : 521<br />
6 threads : 576<br />
7 threads : 619<br />
8 threads : 668<br />
<br />
Kontraktor with dedicated assignment of threads to Actors (see commented line in source above)<br />
<br />
1 threads : 1520<br />
2 threads : 804<br />
3 threads : 571<br />
4 threads : 459<br />
5 threads : 457<br />
6 threads : 534<br />
7 threads : 615<br />
8 threads : 659<br />
<div>
<br /></div>
<div>
<b><span style="font-size: large;"><br /></span></b></div>
<div>
<b><span style="font-size: large;">Conclusion #2</span></b></div>
<div>
<b><span style="font-size: large;"><br /></span></b></div>
<div>
Differences in runtimes can be attributed mostly to the delay in scaling up. For deterministic load problems, prearranged Actor scheduling is more efficient ofc. However thinking of a server receiving varying request loads over time, automatic scaling is a valid option.</div>
<div>
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com25tag:blogger.com,1999:blog-4356402223581768063.post-79509533961097556482014-09-27T01:11:00.000+02:002014-09-27T01:18:55.595+02:00Breaking the habit: Solving the Dining Philosopher's problem with ActorsConcurrent programming using Actors is different to traditional mutex based approaches, therefore I'd like to show how the solution of a classical concurrency problem can be done using Actors. It requires some time to get used to it as future/actor based concurrency looks different and uses different patterns.<br />
<br />
Additionally I'll show how little effort is required to make this a distributed application, as asyncronous Actor based concurrency is much more resilient regarding network induced latency.<br />
<br />
I'll use my Kontraktor Actor library version 2.0 beta to also illustrate how the "message == asynchronous method call" approach of Kontraktor works out in "practice".<br />
<br />
<b>Resources:</b><br />
<br />
Kontraktor lib: <a href="https://github.com/RuedigerMoeller/kontraktor" target="_blank"><b>https://github.com/RuedigerMoeller/kontraktor</b></a>.<br />
Full source of sample is <a href="https://github.com/RuedigerMoeller/kontraktor/blob/trunk/src/examples/java/org/nustaq/kontraktor/examples/Dining.java" target="_blank"><b>here</b></a>.<br />
<br />
Description of the philosophers dining problem:<br />
<a href="http://en.wikipedia.org/wiki/Dining_philosophers_problem"><b>http://en.wikipedia.org/wiki/Dining_philosophers_problem</b></a><br />
(I am using a non-fair solution to keeps things simple.)<br />
<br />
<br />
<b><span style="font-size: x-large;">The Table</span></b><br />
<br />
One actor represents the table. For each fork, a list of philosophers waiting for "fork is free"-notification is kept.<br />
<br />
Short explanation: In Kontraktor all public methods are translated into asynchronous messages put on to an actors queue ("mailbox") behind the scenes. The worker thread of an actor takes the messages of the mailbox and calls the "real" implementation. So its guaranteed an actor is executing single threaded. To identify asynchronous calls easily, I always put a $ in front of public actor methods.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWuJW9wZ4rjL8LzUn7N3IfonFSKTnem0VzEePLgKdgKmyxsYF4uaU04a_JXc3bOG_fX1U07alZPZd2nnyMa35m0l-1fp02ItI51bKYHhXfph8-etoyx9KRVmk8AvIbC6hviaJQ19qBKqA/s1600/Screenshot+from+2014-09-26+21:30:49.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWuJW9wZ4rjL8LzUn7N3IfonFSKTnem0VzEePLgKdgKmyxsYF4uaU04a_JXc3bOG_fX1U07alZPZd2nnyMa35m0l-1fp02ItI51bKYHhXfph8-etoyx9KRVmk8AvIbC6hviaJQ19qBKqA/s1600/Screenshot+from+2014-09-26+21:30:49.png" /></a></div>
<br />
Explanation: if a fork is taken and the queue of waiting philosophers is empty, a fulfilled promise is returned, else an unfulfilled promise is added to the queue. The caller code then looks like:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikuAnPmNwke8KZq2Ee6L0nHPBc8bzVRepau6NM7zeFxH7a-uO7kvOsoe7fwxMPdUrBvyFztmjebxCs4BA6YYVoCTfvpQASnwlxGutaAT7Ifz8Nxj0kQZSL1xjYNpJzxTeAk7KUqdQV8dg/s1600/Screenshot+from+2014-09-26+21:36:39.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikuAnPmNwke8KZq2Ee6L0nHPBc8bzVRepau6NM7zeFxH7a-uO7kvOsoe7fwxMPdUrBvyFztmjebxCs4BA6YYVoCTfvpQASnwlxGutaAT7Ifz8Nxj0kQZSL1xjYNpJzxTeAk7KUqdQV8dg/s1600/Screenshot+from+2014-09-26+21:36:39.png" /></a></div>
<br />
Once a fork is returned, the next Promise in the arraylist (if any) is notified. signal() is equivalent to receive("dummy",null).<br />
<br />
Note that the "result" object is unused, I just need a signal from the future returned. That's why the fulfilled Promise gets a "void" as a dummy result object (which will then trigger immediate execution of the then-closure on caller side).<br />
<br />
Note how I can use simple single threaded collections and data structures as the actor is guaranteed to be executed single threaded.<br />
<br />
<b><span style="font-size: x-large;">The Philosopher</span></b><br />
<br />
A philosopher has a wonderful life which can be divided into 3 states:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">10 <b>think</b> randomTime</span><br />
<span style="font-family: Courier New, Courier, monospace;">20 be_<b>hungry</b>_and_try_to_grab_2_forks</span><br />
<span style="font-family: Courier New, Courier, monospace;">30 <b>eat</b> randomTime</span><br />
<span style="font-family: Courier New, Courier, monospace;">40 <b>goto</b> 10</span><br />
<br />
[rumors are of lost lines 15 and 35 related to indecorous social activities introduced after Epicurus happened to join the table]<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWNGCJcH6sqYyGOeXgS5P_xDFEWReQfUVXr1utuFlG4PzPD2Nxwa5bRw9-JNvA9HyO9H6NosV6RGDnzyTH2r1tuus6HyCExPLwvrC9ihAISIlCWNGHTKd5vFpexwhuSNA0AFiWZ3Tg68s/s1600/Screenshot+from+2014-09-26+22:32:50.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWNGCJcH6sqYyGOeXgS5P_xDFEWReQfUVXr1utuFlG4PzPD2Nxwa5bRw9-JNvA9HyO9H6NosV6RGDnzyTH2r1tuus6HyCExPLwvrC9ihAISIlCWNGHTKd5vFpexwhuSNA0AFiWZ3Tg68s/s1600/Screenshot+from+2014-09-26+22:32:50.png" /></a></div>
<br />
The meat is in the $live method:<br />
After thinking, the philosopher tries to grab one fork, the closure given to the "then" method gets executed once the fork becomes available. Once both forks have been grabbed, another "delayed" call keeps the "eat" state for a while, then both forks are returned and a $live message is put onto the actors mailbox (=message queue) to start thinking again.<br />
<br />
Remarks:<br />
<br />
<ul>
<li>"delayed" just schedules the given closure onto the actors mailbox with a delay.</li>
<li>"self().$live()" puts a $live message onto the actors mailbox. If "this.$live()" would be called, the $live() message would be executed directly [synchronous] like a normal method. In this example this would not hurt, however in many cases this makes a difference.</li>
<li>Wait-Free. The executing thread is never blocked. One could easily simulate thousands of philosophers onto a single thread</li>
</ul>
<br />
<b><span style="font-size: x-large;">Setup and run</span></b><br />
<br />
The main method to start the life of 5 philosophers looks like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmtBnFcCfSDXHAQAWMnLcKn4B8ijDC161CFTozab1942b9rHlnTFRSCXxw_3mybGi-Ad4ptsY2cHcUydAARKrBkNdcMJ3tIO1sv8yIi64vPDK8wg5G_Bjm4L739AgOBw05hQN4FD_G1z8/s1600/Screenshot+from+2014-09-26+23:01:38.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmtBnFcCfSDXHAQAWMnLcKn4B8ijDC161CFTozab1942b9rHlnTFRSCXxw_3mybGi-Ad4ptsY2cHcUydAARKrBkNdcMJ3tIO1sv8yIi64vPDK8wg5G_Bjm4L739AgOBw05hQN4FD_G1z8/s1600/Screenshot+from+2014-09-26+23:01:38.png" /></a></div>
and<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWgCjj2aKzTNTQ8xecBTEri2w4ulQpndwiV8yZ3aTzhEm4RJ7d84PMhifnUGz55iuZuV5i4qEBMLzNw-vIpLEf2Eo44SHFIlPGhCI0K36SgVm7m3fTGGz8y1G19_YkmIrgsTJoL5UGZuQ/s1600/Screenshot+from+2014-09-26+23:03:09.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWgCjj2aKzTNTQ8xecBTEri2w4ulQpndwiV8yZ3aTzhEm4RJ7d84PMhifnUGz55iuZuV5i4qEBMLzNw-vIpLEf2Eo44SHFIlPGhCI0K36SgVm7m3fTGGz8y1G19_YkmIrgsTJoL5UGZuQ/s1600/Screenshot+from+2014-09-26+23:03:09.png" /></a></div>
<br />
Note Hoarde is a utility to create and manage groups of Actors. "startReportingThread" just starts a thread dumping the state of all philosophers:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAGZuHNINDSwx1INjr0_zfCbDU9BGq1yr-DKSpgnVB7nk4hzvXZE-E2g0672InkKdbWFZ4S2F9QIlwLY4BuAmcvO0iEJ-6hjwpbjN7By0F63zIrgDpaXjqm6P76165WBL-5OjOdYNgP5w/s1600/Screenshot+from+2014-09-26+23:18:14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAGZuHNINDSwx1INjr0_zfCbDU9BGq1yr-DKSpgnVB7nk4hzvXZE-E2g0672InkKdbWFZ4S2F9QIlwLY4BuAmcvO0iEJ-6hjwpbjN7By0F63zIrgDpaXjqm6P76165WBL-5OjOdYNgP5w/s1600/Screenshot+from+2014-09-26+23:18:14.png" /></a></div>
<br />
"Is this still Java?" .. probably weird looking stuff for the untrained eye ..<br />
(note Philosopher.$getState is async and yield() is used to wait for all futures to complete before executing the then() closure printing the state)<br />
<br />
BTW: spacing is somewhat inconsistent as I started to reformat code partially during edit .. but won't do screenshots again ..<br />
<br />
Yay, done:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeLacSprpIAQqrIuqmmWJVRWxKPCvXwb5lj7ThKiYpRlwFFsIF6lbVekilpe-mLAWlIVEJXDdBmnaeGphHMwmxWjN81hBg2oVn_jENahj4H0rY31GFYKV_D8OPOs0dP_6nx8Tgv-fKDs0/s1600/Screenshot+from+2014-09-26+23:11:36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeLacSprpIAQqrIuqmmWJVRWxKPCvXwb5lj7ThKiYpRlwFFsIF6lbVekilpe-mLAWlIVEJXDdBmnaeGphHMwmxWjN81hBg2oVn_jENahj4H0rY31GFYKV_D8OPOs0dP_6nx8Tgv-fKDs0/s1600/Screenshot+from+2014-09-26+23:11:36.png" /></a></div>
<br />
<br />
<b><span style="font-size: x-large;">Let's make it a distributed application</span></b><br />
<br />
The killer feature of Actor based asynchronous+lock/wait free programming is simple distributability. A solution using blocking mutexes would have to get a rewrite or rely on synchronous remote calls, so the bigger the network latency, the lower the throughput. Actors do not suffer from that.<br />
We can just change the startup to make the table run remotely:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhY0ZD9SDxvIANy3Jr-NpEpYbCpGWwBQH9zlRpBu55nez6ScDSecQtfz0tpqbzAqkhuhrFhbvRXdLTD-p7OlxLdngNkJDyTNfsL0JnhEItuyfZlXIUnJISjuCUHBAsOnfCwIrfBytG4Rs0/s1600/Screenshot+from+2014-09-26+23:27:55.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhY0ZD9SDxvIANy3Jr-NpEpYbCpGWwBQH9zlRpBu55nez6ScDSecQtfz0tpqbzAqkhuhrFhbvRXdLTD-p7OlxLdngNkJDyTNfsL0JnhEItuyfZlXIUnJISjuCUHBAsOnfCwIrfBytG4Rs0/s1600/Screenshot+from+2014-09-26+23:27:55.png" /></a></div>
<br />
Is this cool or what ? I now can feed hoardes of philosophers with a single table server:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbJrBQSDJlEfk1wVG3_A6mQ6wV-JZT71rb9kKsBwsnAl9q2JeEcaB0_snhFMQfVZFbnywnIbgA6QYrkWR2LRW-riTtfwWI4TQLJfB5DLtMWehgMQ06QWevh9BZaDoVSf6CE6yZ0AK92iw/s1600/Screenshot+from+2014-09-26+23:33:00.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbJrBQSDJlEfk1wVG3_A6mQ6wV-JZT71rb9kKsBwsnAl9q2JeEcaB0_snhFMQfVZFbnywnIbgA6QYrkWR2LRW-riTtfwWI4TQLJfB5DLtMWehgMQ06QWevh9BZaDoVSf6CE6yZ0AK92iw/s1600/Screenshot+from+2014-09-26+23:33:00.png" /></a></div>
<br />
<span style="font-size: large;"><b><br /></b></span>
<br />
<br />Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com37tag:blogger.com,1999:blog-4356402223581768063.post-46150314219416390922014-08-30T16:13:00.000+02:002014-08-30T20:33:14.438+02:00Backpressure in async communication systemsI recently saw this <a href="http://www.infoq.com/presentations/reactive-steams-akka" target="_blank">excellent presentation video</a> of Mr Kuhn .. and later on found about the community standardization effort of <a href="https://github.com/reactive-streams/reactive-streams/blob/v0.3/tck/src/main/resources/spec.md" target="_blank">reactive streams</a>.<br />
<br />
As I have done many experiments dealing with backpressure issues in a high volume async distributed system, I'd like to point out some possible pitfalls of the "flow control" mechanism proposed in the reactive streams spec and point out how I dealt with the problem (after having tried similar solutions).<br />
<br />
The problem of flow control arises in asynchronous communication if the sender sends messages at a higher pace than (one of the) receivers can process them.<br />
This problem is well known from networking and actually TCP implements a similar scheme to the one proposed in the RStreams spec. Another - less known - flow control scheme is NAK. NAK is used typically in low latency brokerless (multicast) messaging products.<br />
<br />
The "Reactive Streams" spec proposes an acknowledge based protocol (although its not presented as such).<br />
<br />
<blockquote class="tr_bq">
Receiver sends "requestMore(numberOfMsg)" to Sender<br />
Sender sends "numberOfMsg" messages to Receiver</blockquote>
<br />
by emitting "requestMore", a Receiver signals it has received+processed (or will have finished processing soon) previously received messages. Its an acknowledgement message (ACK).<br />
<br />
<span style="font-size: large;"><b>Problematic areas:</b></span><br />
<br />
<ol>
<li>it works for point to point communication only. If you have a multicast situation (e.g. one sender multicasts to N receivers), this does not work well. <br />Reason: <br />In order to make this work, the sender needs to be aware of all receivers, then wait for every receiver to have sent a "requestMore", then compute the minimum of all "requestMore" messages and finally send this amount of messages. Once a receiver dies, no "requestMore" is send/received, so the sender can't sent anything for an extended amount of time (receiver-timeout). If one receiver pauses (e.g. GC's), a "requestMore" is missing, so the sender needs to stall all communication.<br />You can see those effects in action with JGroups using a configuration with credit based flow control protocol. Once a cluster node terminates, throughput falls to zero until the node-leave timeout is reached (similar issues are observable with the ACK-based NACKACK reliable UDP transport of JGroups). ACK ftl :-)</li>
<li>The actual throughput depends hard on connection latency. As the ACK signal ("requestMore") takes a longer time then to reach the sender, the receiver buffer must be enlarged depending on latency, additionally the size of chunks requested by "requestMore" must be enlarged. If one does not do this, throughput drops depending on latency. Short said:<br />sizeOfBuffer needs to be >= eventsProcessable per roundTripLatency interval. This can be solved by runtime introspection, however in practice this can be tricky especially with bursty traffic.</li>
<li>As long reliable communication channels (in memory, TCP) are used: Those transports already have implemented a flow control / backpressure mechanism, so "Reactive Streams" effectively doubles functionality present at a lower and more efficient level. Its not too hard to make use of the backpressure signals of e.g. TCP (one solution is an "outbound" queue on sender side, then observe its size). One can achieve the same flow control results without any need for explicit application level ACK/Pull messages like "requestMore". For inmemory message passing, one simply can inspect the queue size of the receiver. [you need a queue implementation with a fast size() implementation then ofc]</li>
</ol>
<div>
<br /></div>
<span style="font-size: large;"><b>Alternative: NAK instead of ACK, adaptive rate limits</b></span><br />
<div>
<br /></div>
<div>
The idea of NAK (=negative acknowledgement) is raise backpressure signals from receiver side only in case a component detects overload. If receivers are not in an overload situation, no backchannel message traffic is present at all.</div>
<div>
<br /></div>
<div>
The algorithm outlined below worked for me in a high throughput distributed cluster:</div>
<div>
<br /></div>
<div>
Each sender has a send rate limit which is increased stepwise over time. Usually the starting default is a high send rate (so its actually not limiting). Receivers observe the size of their inbound queue. Once the queue size grows to certain limits, receivers emit an NAK( QFillPercentage ) message. E.g. 50%, 75%, 87%, 100%. The sender then increases the send rate limit depending on the "strength" of the NAK message. To avoid permanent degrade, the sender increases the send rate stepwise (the step size may depend on the time no NAK was received).</div>
<div>
<br /></div>
<div>
This works for 1:1 async communication patterns as well as for 1:N. In an 1:N situation the sender reacts to the highest strength NAK message within a timing window (e.g. 50ms). The 0-traffic-on-dying-receiver problem is not present. Additionally the sender does not need to track the number of receivers/subscribers, which is a very important property in 1:N multicast messaging.</div>
<div>
<br /></div>
<div>
Finding the right numbers for NAK-triggers, and sender speed up steps can be tricky as they depend on the volatility of sendrate and latency of transport link. However it should be possible to adaptively find appropriate values at runtime. In practice I used empirical values found by experimenting with the given system on sample loads, which is not a viable solution for a general implementation ofc.</div>
<div>
<br /></div>
<div>
The backpressure signal raised by the NAK messages can be applied to the sender either blocking or nonblocking. In concrete system I just blocked the sending thread which effectively puts a dynamically computed (current rate limit) "cost" for each async message send operation. Though this is blocking , it did not hurt performance, as there was no "other work" to do in case a receiver can't keep up. However backpressure signals can be applied in a nonblocking fashion as well ("if (actualSendRate < sendRateLimit) { send } else { do other stuff }").</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br />
<br />
<br />
<br />
<br />
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com34tag:blogger.com,1999:blog-4356402223581768063.post-50047402054559657022014-06-05T20:24:00.000+02:002014-11-23T12:01:56.250+01:00Java Concurrency: Learning from Node.js, Scala, Dart and Go<br />
<b>Preface</b><br />
<br />
Stuff below regarding asynchronism isn't exactly new, however in the Java-world you'll still see a lot of software using synchronous, blocking multi threading patterns, frequently built deep into the architecture of apps and frameworks. Because of the Java tradition to unify concurrency and parallelism, frequently even senior developers are not aware of the fact, one cannot build well scaling high throughput distributed systems using typical java synchronous multi-threading idioms.<br />
<br />
<b><span style="font-size: x-large;">So what is the problem with synchronous processing ?</span></b><br />
<b><br /></b>
Think of a web server handling user requests. In this example, to process a client request, multiple requests to another remote service have to be done.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRrpjmlxUG0R-iaYcAGYKvQ5s4UwMipDqRoJkb8NiJmtKSuWtC9R1THySBr9KfIJiZLul2Eu1_jnRkDGlJonof61vT2mhyswCDgQtH2rHJKQWOiR_ksPqNjPCFEL5S2qTFtTya9EDTVx4/s1600/serverservice.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRrpjmlxUG0R-iaYcAGYKvQ5s4UwMipDqRoJkb8NiJmtKSuWtC9R1THySBr9KfIJiZLul2Eu1_jnRkDGlJonof61vT2mhyswCDgQtH2rHJKQWOiR_ksPqNjPCFEL5S2qTFtTya9EDTVx4/s1600/serverservice.png" height="220" width="320" /></a></div>
<br />
Now, the service can process 10.000 requests per second, but request latency is 5ms (network time, de/encoding). So a simple request/response takes 5+5 ms (back and forth) = 10 ms. A single threaded client implemented in a blocking fashion can send/receive max. 100 requests per second, therefore is unable to saturate the service.<br />
If the webserver is expected to emit on average 1 service requests per client request, its thread pool has to be sized to 100 worker threads in order to maximize throughput and saturate the service.<br />
<br />
Now consider that the remote service makes use of another service with same throughput and latency characteristics.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnvhbjMffCPa4SI84Lsk5yl_6V7fn1Mn8ZwF195WuYTmodXRT1XKRmgSzbvU1blRvx2DDMD6r6qvRHW-TjYYYR4S__ufg_ylnlEs5Cu02ud-xGEIDBMg09ugWlrp78ekyerFdGCcoyDZk/s1600/Screenshot+from+2014-06-01+15:17:11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnvhbjMffCPa4SI84Lsk5yl_6V7fn1Mn8ZwF195WuYTmodXRT1XKRmgSzbvU1blRvx2DDMD6r6qvRHW-TjYYYR4S__ufg_ylnlEs5Cu02ud-xGEIDBMg09ugWlrp78ekyerFdGCcoyDZk/s1600/Screenshot+from+2014-06-01+15:17:11.png" height="228" width="320" /></a></div>
<br />
This increases latency (as seen from the webserver) by another 10 ms, so overall latency of a service request/response is now 20ms in turn requiring a worker pool of 200 threads in the webserver, 100 service threads in service (A).<br />
<br />
It is clear, in a complex distributed system this imposes a lot of problems when using thread based synchronous processing. A change in the infrastructure (=latency changes) might require reconfiguration of many servers/services.<br />
<br />
A common solution to this dilemma is to just configure way too much worker threads in turn getting into multithreading related issues without need. Additionally, <b>a thread is quite a heavy weight object.</b> Its impossible to scale up to say 100.000 or more threads in case. The higher the latency, the more threads must be used to saturate a remote service (e.g. think of inter-datacenter connections), so if you need to process e.g. 50k requests per second and need to query a remote service for each incoming requst, you simply cannot use a synchronous, blocking style. Additionally, having more threads than CPU cores hampers overall performance (context switches), so having 100's of threads per CPU actually will reduce throughput significantly.<br />
<br />
If you think of higher numbers (e.g. 100k requests per second), its clear that one cannot compensate for synchronous, blocking programming by adding more threads.<br />
<br />
<blockquote class="tr_bq">
<i><b>With synchronous IO programming, latency has direct impact on throughput</b></i></blockquote>
<div>
<br />
<br />
<b><span style="font-family: inherit; font-size: x-large;">No problem, we can go asynchronous, we have threads ..</span></b><br />
<br />
(note: sleep is a placeholder for some blocking IO or blocking processing)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTuAnt2RAAqeHpC-nyakauQPprsYKVW3oWRtXZIUXGSdKpWxm3FvFPP1e9L0bgu5AvvUFBBNEFbhoZm-XHZ3Y12gVckusTSJGAJo8cUcYz1Z6YNmXl9vHJRNWSiydYZgwiWO3gsR3WNfA/s1600/bang.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTuAnt2RAAqeHpC-nyakauQPprsYKVW3oWRtXZIUXGSdKpWxm3FvFPP1e9L0bgu5AvvUFBBNEFbhoZm-XHZ3Y12gVckusTSJGAJo8cUcYz1Z6YNmXl9vHJRNWSiydYZgwiWO3gsR3WNfA/s1600/bang.png" /></a></div>
Bang ! because:<br />
<ul>
<li>HashMap must be changed to ConcurrentHashMap, else you might run into endless loop when calling "get" concurrenlty to a "put" operation.</li>
<li>You might see or not the result of "someInt++" (JMM). Needs to be declared 'volatile'.</li>
</ul>
Congrats ! Now most of your code is multithreaded, which means you'll clutter your code with ConcurrentHashMap's and Atomics, synchronized and deadlocks ... the step-up of "callback hell" is "multithreaded callback hell".<br />
<br />
<blockquote class="tr_bq">
<i><b>With plain java, asynchronous programming provokes unnecessary multithreading.</b></i></blockquote>
<br />
A thread-safe version of this code would manage to deliver the callback "inside" the calling thread. But for that we need to enqueue the callback and need kind of a threading framework which ensures the queue is read and executed.<br />
<br />
Well, one can do this in plain java like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhdzkKQay-zNaQHN4ZqXhyJAQ-oJrU8wiSc6lMW9xqogXB0UkJCjvfQCke0xPLaZ58A1mb_dG8QIiMrTbqMACT0ug26ozZ_ZfXjcbO06Cp_ghL8MEbN8wl_35Ie8Oy9lEWJ75iWLuZq5M/s1600/nodjs1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhdzkKQay-zNaQHN4ZqXhyJAQ-oJrU8wiSc6lMW9xqogXB0UkJCjvfQCke0xPLaZ58A1mb_dG8QIiMrTbqMACT0ug26ozZ_ZfXjcbO06Cp_ghL8MEbN8wl_35Ie8Oy9lEWJ75iWLuZq5M/s1600/nodjs1.png" /></a></div>
<br />
Hm, some boilerplate code here, basically a handcrafted simple implementation of an Actor. Actually its a very simple case, as the async call does not return values. In order to realize this, more handcrafting would be required (e.g. a callback interface to deliver the result of blocking code to the calling thread, additionally you won't do the result-callback from your one-and-only-unblockable mailbox processor thread. Tricky stuff ahead.<br />
<br />
With an actor library, this stuff is solved inside the actor framework. Same code with kontraktor:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5CNjC2xsLWlyk8sACE-ruqgBHyn7atv9SaEllh30hL5E0ccNmBq8EUnG4Pec_c2AJ9FNN-Wbtf_Dg19DuEFwQAEQd_30enN6PUjOB1o-ue0KdRgaDNbgauPo5A0Ys0KtzGeOypCuuCoY/s1600/nodjsact.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5CNjC2xsLWlyk8sACE-ruqgBHyn7atv9SaEllh30hL5E0ccNmBq8EUnG4Pec_c2AJ9FNN-Wbtf_Dg19DuEFwQAEQd_30enN6PUjOB1o-ue0KdRgaDNbgauPo5A0Ys0KtzGeOypCuuCoY/s1600/nodjsact.png" /></a></div>
<br />
(I left some comments as well, to avoid dillusion of the fact that only 2-3 lines of extra code are required compared to a synchronous implementation).<br />
<b><br /></b>
<b>Reminder: <a href="http://geekandpoke.typepad.com/.a/6a00d8341d3df553ef017c31a16a73970b-pi" target="_blank">Bad Threads</a></b><br />
<b><br /></b>Using multi threaded asynchronous processing easily leads to massive multi threading, which raises a bunch of problems:<br />
<ol>
<li>Out-of-Control Threading: its hard to tell which parts of your application data/resources are accessed concurrently as software grows (an innocent call to your application framework from a within a foreign callback thread can literally make everything multithreaded unnoticed).</li>
<li>Because of (1) developers start synchronizing defensively</li>
<li>Because of (2) you get deadlocks</li>
<li>Because of (2) you run 500 Threads all waiting on contended locks, so your service goes down despite having just 60% CPU usage.</li>
<li>Because of (1) you get spurious unreproducable errors caused by random cache coherency/data visibility issues.</li>
</ol>
Anyway, lets have a look what newer languages bring to the table regarding concurrency ..<br />
<br />
<br />
<br /></div>
<b><span style="font-size: x-large;">Concurrency Idioms/Concepts in Go, Dart, Node.js, Scala</span></b><br />
<br />
<div>
Despite the hype and discussions surrounding those emerging languages, they technically present more or less the same basic concept to deal with concurrency (ofc there are lots of differences in detail, especially Go is different):</div>
<div>
<blockquote class="tr_bq">
<i><b>Single threaded entities (with associated thread-local state) sequentially process incoming events from bounded or unbounded queue's.</b></i></blockquote>
This technical foundation can be presented in different ways to a programmer. With actors, functionality and data is grouped 'around' the queue ('mailbox'). In the channel model, larger entities ('Process', 'Isolate') explicitely open/close 'channels' (=queues=mailboxes) to exchange messages with other processing entities.<br />
<br />
<br />
<b><span style="font-size: large;">node.js/javascript</span></b><br />
<br />
A node.js process is a single actor instance. Instead of a 'mailbox' the queue in node.js is called "event queue". Blocking operations are externalized to the javascript V8 VM, the results of such an asynchronous operation is then put onto the event queue. The programming model is callback/future style however there are extensions providing a fiber like (software threads, "green" threads) programming model.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9miDRZrJjxajUeUaAH8ogZ_Drq61Ks0bOAehDgK8hled5JqUK6vMiLz_FMOQHg7l16WG644mwhsjEEw8YerJ_IXRLaEFxN077IMeLE2tzOQPbwu3wI605FT6Sxvj0fGEuo2F2YNOOLzY/s1600/jsasync.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9miDRZrJjxajUeUaAH8ogZ_Drq61Ks0bOAehDgK8hled5JqUK6vMiLz_FMOQHg7l16WG644mwhsjEEw8YerJ_IXRLaEFxN077IMeLE2tzOQPbwu3wI605FT6Sxvj0fGEuo2F2YNOOLzY/s1600/jsasync.png" /></a></div>
<br />
Multicore hardware is saturated by running several instances of node.js.<br />
<br />
<b><br /></b>
<span style="font-size: large;"><b>Dart</b> </span><br />
<br />
(on server side) features a more sophisticated, but similar model compared to JS. In contradiction to javascript, Dart provides a richer set of utilities to deal with concurrency.<br />
Its possible to run several event loops called 'Isolates' from within a single process or in separate processes. Conceptually, each isolate is run by a separate thread. Isolates communicate by putting messages onto each others event loop. Very actor'ish, in fact this can be seen as an implementation of the Actor-pattern.<br />
As a special, a Dart <strike>Actor</strike> Isolate reads from 2 event queues, of which one has higher priority and (simplified) processes self-induced asynchronous code, while the other one gets events from the 'outside' world.<br />
<br />
Api-wise, Dart favours 'Futures' over 'Callbacks':<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi99v2HNbt1EtByXibpxbHTawE9JZTUeoN4-3qWyNx_axJizgKrqp9bEw-ftw6xdXxghg7Su6kJYDc5PU4IAfneXn-uNPgsHYqoIXRazDZv8NozbTXzvWsrumr-eQYf8lUw1NK2YpkGaEU/s1600/dartfut.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi99v2HNbt1EtByXibpxbHTawE9JZTUeoN4-3qWyNx_axJizgKrqp9bEw-ftw6xdXxghg7Su6kJYDc5PU4IAfneXn-uNPgsHYqoIXRazDZv8NozbTXzvWsrumr-eQYf8lUw1NK2YpkGaEU/s1600/dartfut.png" height="111" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Ordering in Dart</i></td></tr>
</tbody></table>
<br />
ordered execution of asynchronous pieces of code then can be chained using 'then'.<br />
<br />
For messaging amongs Isolates, Dart provides generic deep-copy of arbitrary object graphs in-process, and simple datatypes for communication amongst isolates running in different processes.<br />
<br />
<blockquote class="tr_bq">
<i><b>Both Dart and javascript actually use a flavour of the Actor pattern, though they use a different terminology.</b></i></blockquote>
<br />
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b>Scala/Akka</b></span><br />
<br />
Although Scala has access to the same concurrency primitives as Java (runs on the JVM), Scala has a tradition of asynchronism, which is reflected by existence of many async tools, idioms and libraries. Its a very creative, adventurous and innovative platform, which can be both good or bad in industrial-grade software projects.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqlZ0_tgA5IH4kcuirFRpewjNClTlzGuk3TgkIiRWgnXaBb_YxJS-9kSXQXNF1O18tGVnILj0ElsgjW2FV14Q00Ie2pjHs68injOQ5rSZQse2G2cQ7eUsH9Lh9ULLrO1WuFlGTa2QDgDE/s1600/akka.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqlZ0_tgA5IH4kcuirFRpewjNClTlzGuk3TgkIiRWgnXaBb_YxJS-9kSXQXNF1O18tGVnILj0ElsgjW2FV14Q00Ie2pjHs68injOQ5rSZQse2G2cQ7eUsH9Lh9ULLrO1WuFlGTa2QDgDE/s1600/akka.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Ordering in Akka</i></td></tr>
</tbody></table>
As to be expected, Scala's Akka+async libraries are very rich featured. Akka proposes a somewhat different system-design phliosophy compared to Dart/Nodes.js as they encourage a more fine grained actor design.<br />
Akka promotes an untyped actor model (though typed actors are available) and has features such as actor-induced message reordering/dropping, re-dispatching, scanning the mailbox, etc. As always: great flexibility also paves the way for abusive and hard to maintain programming patterns. Splitting up an application into many actors (fine grained actor design) is advantageous in that a sophisticated scheduler may parallelize execution better. On the other hand this comes at a high cost regarding maintainability and controllability. Additionally, enqueuing a message instead of doing a plain method call has high fixed overhead, so its questionable wether doing fine grained actor design pays off.<br />
<br />
As Akka originally was designed for Scala, it requires quite some boiler-plating when used from java. One either has to write immutable 'message classes' or (with typed actors) has to define an interface along with each actor (because typed actors use java.*.Proxy internally).<br />
<br />
As I showed in a previous post, its not optimized to the bones, however this should not be significant for real world asynchronous IO/messaging centric applications.<br />
<br />
Recent versions improved remoting of actors and also added support for multicast-based messaging (zeroMQ).<br />
<div>
<br />
<b><span style="font-size: large;">Go</span></b></div>
<br />
If you look at typical processing sequences in an actor based asynchronous program, you'll probably notice its kind of a handcrafted multiplexing of concurrently executed statement sequences onto a single thread. I'll call such a sequence 'threadlet' (Edit: rest of the world calls them 'Monad').<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHk6NYH7dMfse18-_1jjHNY19g2YfYHesGOdi4VuVADJvfaIRll9pPHkfefr8lvw3u71Ujks2S1TbvDu7fClfX4CvOjAKxDLE5fnnlF4kD5oDFzKv4MB4dKWGyXy-aAlBeDNDhkX8bPbg/s1600/urlasync.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHk6NYH7dMfse18-_1jjHNY19g2YfYHesGOdi4VuVADJvfaIRll9pPHkfefr8lvw3u71Ujks2S1TbvDu7fClfX4CvOjAKxDLE5fnnlF4kD5oDFzKv4MB4dKWGyXy-aAlBeDNDhkX8bPbg/s1600/urlasync.png" /></a></div>
<br />
Basically myFunc models an ordered sequence of actions ('threadlet'):<br />
<br />
<ul>
<li>getUrl</li>
<li>process contents</li>
<li>return result</li>
</ul>
<br />
Since getting the URL needs some time, it is executed async, so other events can be read from the inbox.<br />
<br />
If the runtime would be aware of "blocking calls" automatically, it could assist by providing the illusion of single threaded execution by automatically splitting a "threadlet" into separate events. This is what Go does:<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">go myFunc() {<br /> String text = getUrl("http://google.com" );<br /> [process text]<br /> return result;<br />}()</span></blockquote>
While one builds up a "virtual stack" of a "threadlet" in an actor based system by chaining futures and callbacks, Go manages to transform a "normal" looking piece of code behind the scenes. For that it builds up a (dynamically sized) stack which replaces Future chaining of the actor based approaches. Its a non-native lightweight thread.<br />
<br />
So this might look very different, its actually a similar concept of multiplexing small snippets of ordered statement sequences onto a single thread. Its similar to "Green Threads" (low overhead software emulated threads). If such a 'goroutine'/'threadlet' needs to wait or is logically blocked, the processing thread does not stop but processes another goroutine (in case of actors: message from mailbox).<br />
<br />
As the Java, Javascript and Dart Virtual Machines have no concept of "coroutines" (=a runnable piece of code with associated stack and state = software thread = green thread) support, a more explicit syntax to emulate green threads is required for asynchronous programming in those lanugages.<br />
<blockquote class="tr_bq">
<i><b>The rationale behind go-routines/actors/future based asynchronism is, that the programmer should specify concurrency of execution. The paralellism/scheduling is handled by the runtime system.</b></i></blockquote>
The Go runtime schedules go routines onto one or more OS threads. Blocking operations are automatically detected (e.g. OS calls), blocking operations such as waiting on messages from a channel/queue actually do not block as the current thread just continues with executing another go-routine.<br />
So from an abstract point of view, go-routines are similar to lightweight threads with built-in awareness of non-blocking message-queues (channels).<br />
<br />
<b>Channels replace mailboxes,futures,callbacks</b><br />
<br />
An actor encapsulates a queue (mailbox), a set of messages 'understood' and private (=thread local) data. In Go, mailboxes are replaced by queue objects called 'Channels' which can be created and deleted dynamically. A go routine might choose to open one or more queues (channels) it is listening and sending to. The size of a queue can be set to 0, which equals synchronous data transfer (actually still non blocking as a go-routine != a thread).<br />
<br />
<b>Shared State</b><br />
<br />
As go-routines might run in parallel on different cores if the runtime scheduler decides so, there might be multithreaded access to shared data structures (if two go-routines are scheduled on different threads).<br />
As far I understood, in those cases manual synchronization is required. I'd see this as a disadvantage compared to the actor model, where it is very clear wether private or shared data gets accessed. Additionally the concept of actor-local data matches current CPU designs, as each core has a non-shared cache.<br />
The actor pattern enforces a thread-data-ownership by design. Astonishingly I did not find something along the lines of a GO memory model comparable to the JMM, which is required when access to shared data from multiple threads is possible.<br />
<br />
See also<i><b> <a href="http://blog.golang.org/concurrency-is-not-parallelism" target="_blank">Concurrency is not parallelism</a></b></i><br />
<i><br /></i>
<i><br /></i><br />
<b>Dart, Node.js have a shared nothing architecture</b><br />
<br />
No data can be shared amongs processes/isolates. In order to share data, messaging must be used (=has to be encoded and copied). That's probably a contributing factor to the rise of no-sql storage like redis or mongodb. They are used both for persistance and to mimic a shared 'Heap'. In order to share complex data structures, kind of serialization has to be used. Frequently simple solutions, such as transmitting primitve types only or JSon are used, which adds quite some overhead to data sharing.<br />
<br />
Other advantages of Shared Nothing<br />
<ul>
<li>single threaded garbage collection</li>
<li>single threaded VM + JIT</li>
<li>No memory model like JMM required</li>
</ul>
<br />
<br />
<b>Performance constraints</b><br />
<b><br /></b>
While single threaded concurrency has many advantages in IO centric, middle ware layer applications, it is a disadvantage when it comes to situations where several cores are used to speed up processing. E.g. it is not possible to implement something like the Disruptor without shared memory and access to native threading/volatile/barriers.<br />
<br />
<b>Why single threaded concurrency is easier than multithreaded concurrency (true parallelism)</b><br />
<b><br /></b>Actor/CSP based concurrency still is challenging at times, however one does not have to deal with the issues arising from memory visibility rules resulting from modern CPU architectures (see JMM). So a synchronous sequence of statments inside an actor can do a simple HashMap put or increment a counter without even thinking of synchronization or concurrent modification.<br />
<br />
<br />
<span style="font-size: x-large;">Scheduling</span><br />
<br />
As stated, actor (and asynchronous programming) splits up execution into smaller pieces of ordered statement sequences. Go does a similar thing with its go-routines.<br />
A challenge to runtime-system implementations is the scheduling of ordered statement sequences ("threadlets") onto "real" OS threads.<br />
<br />
The V8 VM (Node.js) schedules a single event loop (=mailbox) + some helper threads to "outsource" blocking operations. So the user application is run on a single thread.<br />
<br />
The Dart VM provides a more refined system, having a "micro event loop" and an external event loop. Click <a href="https://www.dartlang.org/articles/event-loop/" target="_blank">here for more details</a> .<br />
<br />
Akka has a pluggable scheduling strategy: thread pool based, fork join (applies work-stealing), and pinned (each actor has its own thread). The queue implementation of the mailbox can be choosen, however there is no concept of several queues. Given that the "dual queues" are just an implementation of a "message priority" concept, it is clear similar concepts can be realized with Akka.<br />
<br />
In my pet-library 'kontraktor', I use a <a href="https://raw.githubusercontent.com/RuedigerMoeller/kontraktor/master/doc/img/Reality.png" target="_blank">dual queue approach</a> similar to Dart (by chance). The programmer defines the mapping of actors to native (Dispatcher-) threads. This gives best results regarding cache-locality <b>if </b>done right. A downside is that there is no dynamic adaption in case an applications has varying load patterns.<br />
<br />
Pure work-stealing algorithm might perform good in micro benchmarks, in real applications the penalty of cache misses when scheduling an actor on a different thread could be much higher compared to micro benchmark results.<br />
<br />
In order to implement a self-optimizing dispatcher, a sophisticated runtime analysis is required (e.g. count cache misses of 'threadlets', track communication patterns amongst different actors) as outlined <a href="http://www1.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf" target="_blank">here</a>. AFAIK no VM actually has implemented such sophisticated algorithms, there is ongoing research in the Go community.<br />
<br />
<a href="http://morsmachine.dk/go-scheduler" target="_blank">Further reading on GO's scheduler.</a><br />
<br />
<br />
<span style="font-size: x-large;"><b>Do we all </b><b>have</b><b> to learn Scala, Go or Dart now ?</b></span><br />
<br />
As Java has all the low level concurrency tools at hand, mostly a change in habit and mind-culture is required. A lot of Java APIs and components feature synchronous, blocking interfaces. Avoid them, they will become problematic in the uprising world of globally distributed systems.<br />
<br />
In fact, JVM based languages are at advantage, because they have the <b>freedom to choose</b> or even mix different concurrency models.<br />
One can implement Actors, Futures (nonblocking ones, not the blocking version like java.concurrent.Future) using the JVM's concurrency primitives. With the introduction of the shorthand syntax for anonymous classes (some call them lambdas) asynchronous code does not look as messy as with prior versions of java.<br />
<br />
If I'd be a student, I'd go with Scala. This will also improve your general programming skills a lot and bring in a lot of interesting ideas and design patterns.<br />
If you are in the middle of a java career and already heavily invested, its probably better to port/copy successful design patterns found in other, newer languages.<br />
<br />
A groundbreaking improvement to Java's concurrency toolset would require VM and language improvements:<br />
<br />
<ul>
<li>There has to be a way for the programmer to express concurrent/asynchronous execution, that differs from expressing parallel execution, so the JVM gets a chance to optimize execution of concurrent tasks. Currently, <b>concurrency</b> and <b>parallelism</b> are expressed using threads. </li>
<li>Native support for continuations would be required (probably a very hard requirement, especially regarding HotSpot). There are some implementations using pure bytecode weaving out there (e.g. WebFlow) which unfortunately have pretty high performance costs.</li>
</ul>
<br />
Anyway, with recent language improvements (java 8) callback/futures style is much more manageable than before.<br />
Another solution might come from hardware/OS. If threads could be realized in a much more lightweight way (+ faster context switch), software-level concurrency by multiplexing might not be required anymore.<br />
<br />
<br />
<b><span style="font-size: x-large;">Real world experience</span></b><br />
<span style="font-size: large;"><b><br /></b></span>
We have transformed two processes of a very IO intensive clustered application from traditional multithreading to an actor based approach. Both of them performed an order of a magnitude faster thereafter. Additionally the code base got much more maintainable as the amount of shared data (data being accessed multithreaded) reduced dramatically. The number of threads declined from >300 to 2 (+some pooled helper threads).<br />
<br />
<b>What we changed:</b><br />
<ul>
<li>Did not use threads to model concurrency. Concurrency != parallelism</li>
<li>We used non blocking IO</li>
<li>We used an actor approach (kontraktor). This way an ownership-relation is created inbetween an execution entity (actor) and datastructures.</li>
<li><b>We respected the short version of the "reactive manifest":</b><br /><br /><br /> <span style="font-size: large;">Never block</span></li>
</ul>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com59tag:blogger.com,1999:blog-4356402223581768063.post-38235695072301193832014-01-28T01:01:00.000+01:002014-09-23T19:57:13.692+02:00Comparision of different concurrency models: Actors, CSP, Disruptor and ThreadsIn order to make use of modern multi core/socket hardware, Java offers threads and locks for concurrent programming. It is well known this model suffers of a number of problems:<br />
<ul>
<li>Deadlocks</li>
<li>Bad scaling due to "over-synchronization" or contention on common resources </li>
<li>Errors caused by invalid synchronization are timing dependent and hard to reproduce. It is common they appear under special conditions (load, hardware) in production.</li>
<li>As a software project grows, it gets hard to safely modify the application. A programmer needs global awareness of an applications control flow and data access patterns in order to change the software without introducing errors.</li>
<li>The subtleties of the Java Memory Model are not well known to most Java programmers. Even if JMM is known, incorrect programming isn't obvious and will fail rarely. It can be a nightmare to spot them from production logs of large applications.</li>
</ul>
So I am looking for alternative models to express concurrency. Therefore a simple benchmark of the most popular "alternative" concurrency models is done: Actors, CSP and Disruptor.<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Actors</b></span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://blog.easi.net/dpics/gallery/2013-08-20-01-29-43_full-mailbox.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://blog.easi.net/dpics/gallery/2013-08-20-01-29-43_full-mailbox.jpg" height="150" width="200" /></a></div>
<i>Real Life Analogy: People cooperating by sending mails to each other.</i><br />
<br />
An Actor is like an object instance executed by a single thread. Instead of direct calls to methods, messages are put into the Actors "mailbox" (~queue). The actor single threaded reads and processes messages from the queue sequentially (with exceptions).<br />
<span style="font-size: large;"><span style="font-size: small;">Internal state is exposed/shared by passing messages (with copied state) to other Actors.</span></span><br />
<div>
<span style="font-size: large;"><span style="font-size: small;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://blog.easi.net/dpics/gallery/2013-08-20-01-29-43_full-mailbox.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKQgUXccljaGcmAsJxKqbt71_Xuzsrbslrhn9AK4xdxoBoUoTthKL3E8hiHmSkFkOUODOlOD6vPJ8Afk3xNZN_SMJH9YqjKgEAw4mJMqtlTEHeFF11PGPTWnYqQnG-0n-DQGMror_Ti6s/s1600/actorz.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKQgUXccljaGcmAsJxKqbt71_Xuzsrbslrhn9AK4xdxoBoUoTthKL3E8hiHmSkFkOUODOlOD6vPJ8Afk3xNZN_SMJH9YqjKgEAw4mJMqtlTEHeFF11PGPTWnYqQnG-0n-DQGMror_Ti6s/s1600/actorz.png" height="241" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><br />
<br />
<br /></td></tr>
</tbody></table>
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b>CSP</b> (Communicating Sequential Processes)</span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://cdn-www.cracked.com/articleimages/wong/invent/bell1.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://cdn-www.cracked.com/articleimages/wong/invent/bell1.jpg" height="112" width="200" /></a></div>
<i>Real Life Analogy: People phoning each other.</i><br />
<br />
Though a different terminology is used, CSP systems can be seen as a special actor system having bounded mailboxes (queues) of size 0.<br />
<br />
So if one process (~Actor) wants to pass a message to another process, the caller is blocked until the receiver accepts the message. Alternatively to being blocked, a CSP-process can choose to do other things e.g. check for incoming messages (this introduces non determinism to the order of outgoing messages). A receiver cannot accept incoming messages if he is occupied with processing a prior one.<br />
<br />
<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Disruptor</b></span><br />
<b></b><br />
<b></b><br />
The disruptor data structure came up some years ago and was invented and pioneered by <span class="st"><a href="http://mechanical-sympathy.blogspot.com/" target="_blank">Martin Thompson</a>, <a href="http://bad-concurrency.blogspot.de/" target="_blank">Mike Barker</a>, and <a href="http://www.davefarley.net/" target="_blank">Dave Farley</a> at LMAX exchange</span>.<br />
<br />
<i>Real Life Analogy: Assembly line</i><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYCB49wRgpDkOQ0sn6e0cXqb1XVPD4x7UxT44TY7Z6lqALDMq-Tfh5vxhgyhxppQo2fMkqaoqe3c6O8tfLqnfTbbmgsQ7ALE4kFWZ6WyVBCs0qVRQXyMs6wapYvpTVCTisH8La7F5fMvQ/s1600/assembly.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYCB49wRgpDkOQ0sn6e0cXqb1XVPD4x7UxT44TY7Z6lqALDMq-Tfh5vxhgyhxppQo2fMkqaoqe3c6O8tfLqnfTbbmgsQ7ALE4kFWZ6WyVBCs0qVRQXyMs6wapYvpTVCTisH8La7F5fMvQ/s1600/assembly.png" height="386" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdfwVccc6mOHvoqG5VgDy2TSNKeocwez1YVQoV6OkzL7o3c20TWkjw_blV4qpyQF4JHekjBMzr78jJUlCoUHZVH__d129d8A3MqzNRYKivsgBHirOKWdQme7ljjJruFHrgOqQf5ZkjBmU/s1600/disruptor.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdfwVccc6mOHvoqG5VgDy2TSNKeocwez1YVQoV6OkzL7o3c20TWkjw_blV4qpyQF4JHekjBMzr78jJUlCoUHZVH__d129d8A3MqzNRYKivsgBHirOKWdQme7ljjJruFHrgOqQf5ZkjBmU/s1600/disruptor.png" height="428" width="640" /></a></div>
<br />
<br />
The disruptor is a bounded queue (implemented by a ring buffer) where producers add to the head of the queue (if slots are available, else the producer is blocked).<br />
Consumers access the queue at different points, so each consumer has its own read position (cursor) inside the queue. Sequencing is used to manage queue access and processing order.<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Thread + locking of shared data</b></span><br />
<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="http://csinvesting.org/wp-content/uploads/2013/02/Crowded-Pit.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="http://csinvesting.org/wp-content/uploads/2013/02/Crowded-Pit.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">contention</td></tr>
</tbody></table>
Ok, everybody knows this. Its Java's default to model concurrent execution. Actually these are the primitives (+CAS) used to build higher level concurrent models like those described above.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Conclusion</b></span><br />
<br />
<br />
While the CSP/Actor-Model realizes inter-thread communication by
passing/copying thread-local data to some queue or process, the Disruptor keeps data in place and
assures only one thread (consumer or producer) is owner of a data item
(=queue slot) at a time.<br />
This model seems to fit existing CPU, memory and
VM architectures better resulting in high throughput and superior
performance. It avoids high allocation rates and reduces the
probability of cache misses. Additionally it implicitly balances
processing speed of interdependent consumers without growing queues
somewhere.<br />
There is no silver bullet for concurrent programming, one still has to plan carefully and needs to know what's going on.<br />
<br />
While I am writing this, I realize each of these patterns has its set of best-fit problem domains, so my initial intention using a single benchmark to compare performance of different concurrency models might be somewhat off :-).<br />
<br />
<i></i><br />
<b><br /></b><br />
<span style="font-size: large;"><b>The Benchmark Application</b></span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<b> </b><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheaE9el5yg_eTCcxdXHYU9GhoqvPKb_m_4KJRQ-dZnqJLbax5NMppSvfyOMuBUpWwjybeqnYNAm9Ja_TSzZWYt7iEgHvhdhF3UmyLGH5aFQEf33EQzUdii4kRq6ooNeWNggpthyuHb_5U/s1600/pi.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheaE9el5yg_eTCcxdXHYU9GhoqvPKb_m_4KJRQ-dZnqJLbax5NMppSvfyOMuBUpWwjybeqnYNAm9Ja_TSzZWYt7iEgHvhdhF3UmyLGH5aFQEf33EQzUdii4kRq6ooNeWNggpthyuHb_5U/s1600/pi.png" height="226" width="400" /></a></div>
<br />
<b>Update:</b> Jason Koch implemented a custom executor performing significantly better than the stock JDK executors. See his <b><a href="http://fasterjava.blogspot.de/2014/09/writing-non-blocking-executor.html" target="_blank">blog post</a>.</b><br />
<br />
The benchmark approximates the value of PI concurrently. Therefore N jobs are created, and the result of each job must be added to finally get an approximation of PI. For maximum performance one would create one large job for each Core of the CPU used. With this setup all solutions roughly would perform the same as there is no significant concurrent access and scheduling overhead.<br />
<br />
However if we compute PI by creating 1 million jobs, each computing a 0..100-loop slice of PI, we get an impression of:<br />
<ul>
<li>cost of accessing shared data when the result of each job is added to one single result value</li>
<li>overhead created by the concurrency control mechanics (e.g. sending messages/scheduling Runnables to a thread pool, CAS, Locks, volatile r/w ..).</li>
</ul>
The test is run with 2 configurations<br />
<ul>
<li>100 iterations per pi-slice-job, 1,000,000 jobs</li>
<li>1,000 iterations per pi-slice-job, 100,000 jobs</li>
</ul>
As hardware I use <br />
<ul>
<li>AMD Opteron 6274 Dual Socket 2,2 Ghz. Each socket has 8 cores + 8 Hardware Threads (so overall 16 cores + 16 HW Threads)</li>
<li>Intel XEON@3Ghz dual socket six core (12 cores + Hyperthreading turned off). (2011 release)</li>
</ul>
I never use more threads as there are "real" cores.<br />
<br />
<br />
<br />
<span style="font-size: large;">Stop the blabber, gimme results ! </span><br />
<br />
<br />
See bottom of this post for the benchmark source.<br />
<ul>
<li><b>Threads </b>- somewhat naive thread based implementation of the Pi computation. Enough effort invested it is possible to match any of the other results of course. At the core of the VM, threads, locks (+atomic, volatile r/w + CAS) are the only concurrent primitives. However there is no point in creating an ad-hoc copy of the Disruptor or an Actor system in order to compare concurrency approaches.</li>
<li><b>Akka</b> - a popular Actor implementation on the VM. The benchmark has been reviewed and revised (especially the ActorSystem configuration can make a big difference) by the Akka crew. Threads are scheduled using Java 7's fork join pool. Actually the Pi computation is one of Akka's tutorial examples. </li>
<li><b>Abstraktor</b> - my experimental Actor/CSP implementation. It's using short bounded queues (so leans more to the CSP side) and avoids deadlocks by maintaining 2 queues per Actor (in and out). If the out-queue is blocked, it just reads from the in-queue. <br />I am using Nitsan Wakarts excellent MPSC queue implementation (check his <a href="http://psy-lob-saw.blogspot.de/" target="_blank">blog</a> or github <a href="https://github.com/nitsanw/JAQ-InABox" target="_blank">jaq-in-a-box</a>) and that's the major reason it shows kind of competitive performance+scaling. <br />I use this to get a rough baseline for comparision and experiment with different flavours of Actors/CSP. Probably the only thing one can do with it is to run the Pi bench ;-).<br /><b>Update: </b>The experimental version benchmarked here has been consolidated + improved. You can find it <a href="https://github.com/RuedigerMoeller/kontraktor" target="_blank">o</a>n <a href="https://github.com/RuedigerMoeller/kontraktor" target="_blank">https://github.com/RuedigerMoeller/kontraktor</a></li>
<li><b>Disruptor</b> - my naive approach implementing the benchmark based on the Disruptor 3.2. It turned out that I used a not-yet-polished utility class, however I keep this benchmark just to illustrate how smallish differences in implementation may have big consequences.</li>
<li><b>Disruptor2 </b>- As Michael Barker would have implemented it (Thanks :-) ). Its actually more than twice as fast for the 1 million test as the closest runner up.</li>
</ul>
<b>Intel (Xeon 2 socket each 6 cores, Hyperthreading off)</b><br />
<b> </b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7TzYlqqmG5rE3RIrTYo9_7H__SwJyIuhAYeS0Y5P2biyDg3OppRO4asMfRl6WBxTxiBs42rjYIN20vAqjJUPQHmr-iERIqpiAjHsLw-vBCjKeUosIVAWY8oiRu-yaZ0BE35WLSpPF_5A/s1600/100kintel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7TzYlqqmG5rE3RIrTYo9_7H__SwJyIuhAYeS0Y5P2biyDg3OppRO4asMfRl6WBxTxiBs42rjYIN20vAqjJUPQHmr-iERIqpiAjHsLw-vBCjKeUosIVAWY8oiRu-yaZ0BE35WLSpPF_5A/s1600/100kintel.png" /></a></div>
Well, it seems with 100k jobs of 1000 iterations, the benchmark is dominated by computation, not concurrency control. Therefore I retry with 1 million jobs each computing a 100 iteration slice.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUnL6WuQiIW5PUTGW-sAPhXsTRyKGYRqS2H2OtGtZWziu7bnZEXUe7ViCE4tq8tKHMrGi2LX-OLYtQVQTu1xPqhEy4fna_v6ScCw_AXu4iqDtSLhDkVgcboWe8yRBsTW0TCj7t4oAfhQo/s1600/1mintel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUnL6WuQiIW5PUTGW-sAPhXsTRyKGYRqS2H2OtGtZWziu7bnZEXUe7ViCE4tq8tKHMrGi2LX-OLYtQVQTu1xPqhEy4fna_v6ScCw_AXu4iqDtSLhDkVgcboWe8yRBsTW0TCj7t4oAfhQo/s1600/1mintel.png" /></a></div>
<br />
Ok, we probably can see differences here :-)<b><br /></b><br />
<br />
<br />
<b>AMD Opteron 6274 Dual Socket 2,2 Ghz (=16 real cores, 16 HW Threads)</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbztUpo5GEW5kENOEf_Xq2jU0GBB7Sdsv2EwQsmerUqUNxrqVv90S8kwMZUNZlqEh8raDCbcxV4vFwe6AAE7V8Ys-kk_KmlPB_c-pDX_9-puVInQDnaO5t7PXFIc2dNZ5QX-d6_0UZnZk/s1600/100kopt.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbztUpo5GEW5kENOEf_Xq2jU0GBB7Sdsv2EwQsmerUqUNxrqVv90S8kwMZUNZlqEh8raDCbcxV4vFwe6AAE7V8Ys-kk_KmlPB_c-pDX_9-puVInQDnaO5t7PXFIc2dNZ5QX-d6_0UZnZk/s1600/100kopt.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd7H_I0hHiQ5gmVR808OZt4l4mfe9fiCECliPUdi4LVgGcUIVChw2bIU45TZxodvRIdHPMYN9Zjffhuf0Rdoo_9ik9TmLiTrJtcJQAlwAAhHzO1MwG54Qhiwu-IyxVV1yd0Waz53ZHnY0/s1600/100kopteron.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
Again the 1 million tiny-job variant spreads the difference amongst the approaches (and their implementation):</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLzOiz1odLlHvascWpfmzPqrUTc_BSBrXc2wUkUyhkdcVT3-i6wMDfCtVfUrXNiOBtw6RZuOlflKJU2-5_BnPWkWyFB5q7xH4hZpfmyx-s3U6-OGW-_1CPEHLvW2bG93f1xUddyJQgXFA/s1600/1mopt.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLzOiz1odLlHvascWpfmzPqrUTc_BSBrXc2wUkUyhkdcVT3-i6wMDfCtVfUrXNiOBtw6RZuOlflKJU2-5_BnPWkWyFB5q7xH4hZpfmyx-s3U6-OGW-_1CPEHLvW2bG93f1xUddyJQgXFA/s1600/1mopt.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Note there is like 5% run to run jitter (GC and stuff), however that does not change the big picture.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Last but not least: Best results per CPU architecture per lib:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVhyphenhyphenMZeNTkzdAUPZPyqKGDMxfiXtHb5T58wrtd20n05R_5xxiGvDs6oHUYBtB3TtQkBAT2_6L-itycW7IcIMwEjP4cjjJ1DAnPdmblzf6Ji0SyhRSXxrWiJRUeP7sLXjVJzeYyWr3QuBk/s1600/overall.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVhyphenhyphenMZeNTkzdAUPZPyqKGDMxfiXtHb5T58wrtd20n05R_5xxiGvDs6oHUYBtB3TtQkBAT2_6L-itycW7IcIMwEjP4cjjJ1DAnPdmblzf6Ji0SyhRSXxrWiJRUeP7sLXjVJzeYyWr3QuBk/s1600/overall.png" height="245" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;"><b><span style="background-color: white;">Discussion</span> of Results</b></span></div>
<br />
<br />
We see that scaling behaviour can be significantly different depending on the hardware platform used.<br />
<br />
Akka loses a lot of CPU time doing GC and allocation. If I modify Abstraktor to use unbounded Concurrent Linked Queue (Akka uses those), it performs similar to Akka and builds up temporary queues >100k elements. This is an inherent issue of the Actor model. By leaning towards CSP (use very short bounded blocking queues with size <100), performance also suffers, as threads are blocked/spinning for input too often. However with a queue size of 5000 elements, things work out pretty well (introducing other problems like deadlocks in case of cyclic Actor graphs).<br />
The Disruptor library is very well implemented, so a good part of the performance advantage could be attributed to the excellent quality of implementation.<br />
<br />
Cite from the Disruptor documentation regarding queuing:<br />
<blockquote class="tr_bq">
<i>3.5 The Problems of Queues</i><br />
<i>[...]
If an in-memory queue is allowed to be unbounded then for many classes
of problem it can grow unchecked until it reaches the point of
catastrophic failure by exhausting memory. This happens when producers
outpace the consumers. Unbounded queues can be useful in systems where
the producers are guaranteed not to outpace the consumers and memory is a
precious resource, but there is always a risk if this assumption
doesn’t hold and queue grows without limit. [...]</i><br />
<i>When in
use, queues are typically always close to full or close to empty due to
the differences in pace between consumers and producers. They very
rarely operate in a balanced middle ground where the rate of production
and consumption is evenly matched. [...] </i></blockquote>
Couldn't have said it better myself =).<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Conclusion</b></span> <br />
<br />
<br />
Although the Disruptor worked best for this example, I think looking for "the concurrency model to go for" is wrong. If we look at the real world, we see all 4 patterns used dependent on use case. <br />
So a broad concurrency library ideally would integrate the assembly-line pattern (~Disruptor), queued messaging (~Actors) and unqueued communication (~CSP).<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Benchmark source</b></span><br />
<br />
<br />
<div>
<b>AKKA</b></div>
<div>
<br /></div>
<div>
<div style="height: 400px; overflow: scroll;">
<script src="https://gist.github.com/RuedigerMoeller/8272966.js"></script>
</div>
<br />
<br /></div>
<div>
<br />
<b>Multi Threading</b><br />
<b><br /></b>
<b><br /></b></div>
<div style="height: 400px; overflow: scroll;">
<script src="https://gist.github.com/RuedigerMoeller/8273307.js"></script>
</div>
<br />
<br />
<b>Abstraktor</b><br />
<br />
<br />
<div style="height: 400px; overflow: scroll;">
<script src="https://gist.github.com/RuedigerMoeller/8273494.js"></script>
</div>
<br />
<br />
<b>Disruptor naive</b><br />
<br />
<br />
<div style="height: 400px; overflow: scroll;">
<script src="https://gist.github.com/RuedigerMoeller/8659844.js"></script>
</div>
<br />
<br />
<b>Disruptor optimized</b><br />
<br />
<br />
<div style="height: 400px; overflow: scroll;">
<script src="https://gist.github.com/RuedigerMoeller/8659877.js"></script></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com206tag:blogger.com,1999:blog-4356402223581768063.post-72163629113117314792013-12-21T02:32:00.003+01:002014-08-31T01:02:44.960+02:00Big Data the 'reactive' way<h4>
</h4>
<b>This has been posted (by me ofc) on the Java Advent Calendar originally. Check it out for more interesting articles:</b><br />
<b><br /><a href="http://www.javaadvent.com/2013/12/big-data-reactive-way.html">http://www.javaadvent.com/2013/12/big-data-reactive-way.html</a></b><br />
<div>
<b><br /></b></div>
<div>
A metatrend going on in the IT industry is a shift from query-based, batch oriented systems to (soft) realtime updated systems. While this is associated with financial trading only, there are many other examples such as "Just-In-Time"-logistic systems, flight companies doing realtime pricing of passenger seats based on demand and load, C2C auction system like EBay, real time traffic control and many more.<br />
<br />
It is likely this trend will continue, as <b><i>the (commercial) value of information is time dependent</i></b>, value decreases with age of information.<br />
<br />
Automated trading in the finance sector is just a forerunner in this area, because some microseconds time advantage can be worth millions of dollars. Its natural real time processing systems evolve in this domain faster.<br />
<br />
However big parts of traditional IT infrastructure is not designed for reactive, event based systems. From query based databases to request-response based Http protcol, the common paradigm is to store and query data "when needed".<br />
<br />
<h4>
Current Databases are static and query-oriented</h4>
<div>
<br /></div>
Current approaches to data management such as SQL and NOSQL databases focus on data transactions and <b>static query </b>of data. Databases provide convenience in slicing and dicing data but they do not support update of complex queries in real time. Uprising NOSQL databases still focus on computing a static result.<br />
Databases are clearly not "reactive".<br />
<br />
<h4>
Current Messaging Products provide poor query/filtering options</h4>
<div>
<br /></div>
Current messaging products are weak at filtering. Messages are separated into different streams (or topics), so clients can do a raw preselection on the data received. However this frequently means a client application receives like 10 times more data than needed, doing fine grained filtering 'on-top'.<br />
A big disadvantage is, that the topic approach builts filter capabilities "into" the system's data design.<br />
E.g. if a stock exchange system splits streams on a per-stock base, a client application still needs to subscribe to all streams in order to provide a dynamically updated list of "most active" stocks. Querying usually means "replay+search the complete message history".<br />
<br />
<h3>
A scalable, "continuous query" distributed Datagrid. </h3>
<div>
<br /></div>
I had the enjoyment to do conceptional & technical design for a large scale realtime system, so I'd like to share a generic scalable solution for continuous query processing at high volume and large scale.<br />
<br />
It is common, that real-time processing systems are designed "event sourced". This means, persistence is replaced by journaling transactions. System state is kept in memory, the transaction journal is required for historic analysis and crash recovery only.<br />
Client applications do not query, but listen to event streams instead. A common issue with event sourced systems is the problem of "late joining client". A late client would have to replay the whole system event journal in order to get an up-to-date snapshot of the system state.<br />
In order to support late joining clients, a kind of "Last Value Cache" (LVC) component is required. The LVC holds current system state and allows late joiners to bootstrap by querying.<br />
In a high performance, large data system, the LVC component becomes a bottleneck as the number of clients rises.<br />
<br />
<h4>
Generalizing the Last Value Cache: Continuous Queries</h4>
<br />
In a continuous query data cache, a query result is kept up to date automatically. Queries are replaced by subscriptions.<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"><b>subscribe </b>* from Orders where<br /> symbol in ['ALV', 'BMW'] and<br /> volume > 1000 and<br /> owner='MyCompany'</span></blockquote>
creates a message stream, which initially performs a query operation, after that updates the result set whenever a data change affecting the query result happened (transparent to the client application). The system ensures each subscriber receives exactly the change notifications necessary to keep its "live" query results up-to-date.<br />
<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikwkUyhshTrf4RDyT-RRvHeMuljTyE-UsDmzudcki-z31iaH9BmWWRWSJ1KWQIqW8euykIdBvT7f_Ub3Qi8iO8ZUyDQdkRFv1Xag0_MT5kF7NOjqk-6iwHCSR0wtGkFwNj9OzlXo-xWnk/s1600/Real+Live.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikwkUyhshTrf4RDyT-RRvHeMuljTyE-UsDmzudcki-z31iaH9BmWWRWSJ1KWQIqW8euykIdBvT7f_Ub3Qi8iO8ZUyDQdkRFv1Xag0_MT5kF7NOjqk-6iwHCSR0wtGkFwNj9OzlXo-xWnk/s640/Real+Live.png" height="640" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px; text-align: center;">A distributed continous query system: The LVC Nodes hold data. Transactions are sent to them on a message bus (red). The LVC nodes compute the actual difference caused by a transaction and send change notifications on a message bus (blue). This enables "processing nodes" to keep a mirror of their relevant data partition up-to-date. External clients connected via TCP/Http do not listen to the message bus (because multicast is not an option in WAN). "Subscription processors" keep the client's continuous queries up-to-date by listening to the (blue) message bus and dispatching required change notifications only to client's point2point connection.</td></tr>
</tbody></table>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<b><br /></b><b><br /></b><br />
<br />
<b><br /></b>
<b><br /></b><b><br /></b><b> </b><br />
<b> </b><br />
<b><br /></b><b>Difference of data access patterns compared to static data management:</b><br />
<b><br /></b>
<br />
<ul>
<li>High write volume<br />Real time systems create a high volume of write access/change in data.</li>
<li>Fewer full table scans.<br />Only late-joining clients or changes of a query's condition require a full data scan. Because continuous queries make "refreshing" a query result obsolete, Read/Write ratio is ~ 1:1 (if one counts the change notification resulting from a transaction as "Read Access").</li>
<li>The majority of load is generated, when evaluating queries of active continuous subscriptions with each change of data. Consider a transaction load of 100.000 changes per second with 10.000 active continuous queries: this requires 100.000*10.000 = <b>1 Billion evaluations of query conditions per second</b>. That's still an underestimation: When a record gets updated, it must be tested whether the record has matched a query condition before the update and whether it matches after the update. A record's update may result in an add (because it matches after the change) or a remove transaction (because the record does not match anymore after a change) to a query subscription.</li>
</ul>
<br />
<b>Data Cluster Nodes </b><b>("LastValueCache Nodes")</b><br />
<br />
Data is organized in tables, column oriented. Each table's data is evenly partitioned amongst all data grid nodes (=last value cache node="LVC node"). By adding data nodes to the cluster, capacity is increased and snapshot queries (initializing a subscription) are sped up by increased concurrency.<br />
<br />
There are three basic transactions/messages processed by the data grid nodes:<br />
<br />
<ul>
<li>AddRow(table,newRow), </li>
<li>RemoveRow(table,rowId), </li>
<li>UpdateRow(table, rowId, diff). </li>
</ul>
<br />
The data grid nodes provide a lambda-alike (row iterator) interface supporting the iteration of a table's rows using plain java code. This can be used to perform map-reduce jobs and as a specialization, the initial query required by newly subscribing clients. Since ongoing computation of continuous queries is done in the "Gateway" nodes, the load of data nodes and the number of clients correlate weakly only.<br />
<br />
All transactions processed by a data grid node are (re-)broadcasted using multicast "Change Notification" messages.<br />
<br />
<h4>
Gateway Nodes</h4>
<br />
Gateway nodes track subscriptions/connections to client applications. They listen to the global stream of change notifications and check whether a change influences the result of a continuous query (=subscription). This is very CPU intensive.<br />
<br />
Two things make this work:<br />
<br />
<ol>
<li>by using plain java to define a query, query conditions profit from JIT compilation, no need to parse and interpret a query language. HotSpot is one of the best optimizing JIT compilers on the planet.</li>
<li>Since multicast is used for the stream of global changes, one can add additional Gateway nodes with ~no impact on throughput of the cluster.</li>
</ol>
<br />
<h4>
Processor (or Mutator) Nodes</h4>
<div>
<br /></div>
<div>
These nodes implement logic on-top of the cluster data. E.g. a statistics processor does a continuous query for each table, incrementally counts the number of rows of each table and writes the results back to a "statistics" table, so a monitoring client application can subscribe to realtime data of current table sizes. Another example would be a "Matcher processor" in a stock exchange, listening to orders for a stock, if orders match, it removes them and adds a Trade to the "trades" table.</div>
<div>
If one sees the whole cluster as kind of a "giant spreadsheet", processors implement the formulas of this spreadsheet.</div>
<div>
<br />
<h4>
Scaling Out</h4>
</div>
<div>
<br /></div>
<div>
<ul>
<li>with data size:<br />increase number of LVC nodes</li>
<li>Number of Clients<br />increase subscription processor nodes.</li>
<li>TP/S<br />scale up processor nodes and LVC nodes</li>
</ul>
Of cause the system relies heavily on availability of a "real" multicast messaging bus system. Any point to point oriented or broker-oriented networking/messaging will be a massive bottleneck.<br />
<br />
<br /></div>
<h3>
</h3>
<h3>
Conclusion</h3>
<div>
<br />
Building real time processing software backed by a continuous query system simplifies application development a lot.</div>
<div>
<ul>
<li>Its model-view-controller at large scale.<br />Astonishing: patterns used in GUI applications for decades have not been extended regulary to the backing data storage systems.</li>
<li>Any server side processing can be partitioned in a natural way. A processor node creates an in-memory mirror of its data partition using continuous queries. Processing results are streamed back to the data grid. Computing intensive jobs, e.g. risk computation of derivatives can be scaled by adding processor instances subscribing to distinct partitions of the data ("sharding").</li>
<li>The size of the Code Base reduces significantly (both business logic and Front-End).<br />A lot of code in handcrafted systems deals with keeping data up to date.</li>
</ul>
</div>
<h4>
</h4>
<h4>
About me</h4>
I am a technical architect/senior developer consultant at an european company involved heavily in stock & derivative trading systems.<br />
<br />
<br />
<em>This post is part of the <a href="http://javaadvent.com/">Java Advent Calendar</a> and is licensed under the <a href="https://creativecommons.org/licenses/by/3.0/">Creative Commons 3.0 Attribution</a> license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!</em></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com21tag:blogger.com,1999:blog-4356402223581768063.post-53064353357169626432013-12-15T19:39:00.000+01:002014-10-05T14:14:50.098+02:00Dart - a possible solution to the high.cost@lowquality.webapp syndromeI am involved in development of high quality real time middle ware/GUI front-end applications professionally. Up to now, we were forced to use traditional Fat Client/Server designs, as Web Applications fail to fulfil basic usability requirements (e.g. Key Shortcuts) and performance requirement (high frequency real time updates). Basic UI features often require nasty and unmaintainable JavaScript hacks (the awkward, unfunful kind of hack) which blow up development+testing cost.<br />
<br />
Regarding user experience/performance, checkout this example of a (good!) server centric framework:<br />
<div>
<br /></div>
<div>
<a href="http://www.wicket-library.com/wicket-examples/ajax/tree/simple;jsessionid=1D907A232F857EEC0AA5D7D687D1F143?0">http://www.wicket-library.com/wicket-examples/ajax/tree/simple;jsessionid=1D907A232F857EEC0AA5D7D687D1F143?0</a></div>
<div>
(no offence, Apache-Wicket is one of the best Web-Frameworks out there IMO).</div>
<div>
<ul>
<li>its sluggish</li>
<li>can't navigate with arrow keys</li>
<li>no standard multi selection (with Ctrl, Shift etc.)</li>
</ul>
</div>
if we go down the JavaScript route, we can do better:<br />
<a href="http://www.jqwidgets.com/jquery-widgets-demo/demos/jqxgrid/index.htm?(arctic)#demos/jqxgrid/filtering.htm">http://www.jqwidgets.com/jquery-widgets-demo/demos/jqxgrid/index.htm?(arctic)#demos/jqxgrid/filtering.htm</a><br />
<br />
However (having fiddled with JS myself) its suspicious:<b> one can't resize any of the demo tables</b>. Googling results in the following finding:<br />
<br />
<blockquote class="tr_bq">
<i>The Grid widget works with fixed width only which is specified by its ‘width’ property. It is not possible to set its width in percentages in this version. We’ll implement the requested feature for a future version. </i></blockquote>
<blockquote class="tr_bq">
<i>Best Regards, </i><i>XXXX, DevTeam</i></blockquote>
<br />
So <b>by the end of year 2013, it is still a problem to achieve basic UI functionality with current web standards and browsers</b>. Even worse, it fails to respect basic software engineering principles such as DRY (don't repeat yourself), encapsulation, ..<br />
(check out the "source" tab of the link above to get an impression of the mess required to create a simple table demo component).<br />
<br />
This is not the fault of the libraries mentioned above, the root cause of this mess are the underlying half baked W3C "standards" (+ browser implementations). They are apparently defined by people never involved in real world application development.<br />
<br />
So we sit there with an<b> intact hype, but without a productive development stack </b>to fulfil the vision. Since management tends to share the hype but does not validate technical feasibility, this led to some prominent mis-decisions such as Steve Jobs claiming WebApps can be used on the iPhone instead of native apps (first iPhone release had no native APIs), another example is facebook relying on html5 too much, now moving back to native phone apps hastily. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Following I'll explain why I think we urgently need the <b>paradigm shift </b>in Web-Applications. Without a fundamental change, industry will keep burning money trying to build WebApplications based on expensive , crappy, half-baked technology.<br />
<br />
<h2>
In the beginning there was REST</h2>
and it sucked right away ..<br />
<br />
The use of the <b>REST</b> (<b style="background-color: white; font-family: sans-serif; font-size: 13px; line-height: 19.1875px;">Representational State Transfer) </b>design pattern results in<br />
<br />
<ul>
<li>the absence of state in a server application.<br /><b>Any durable client state gets passed with each request of a client.</b> </li>
<li>each user interaction (e.g. click a button) requires a complete html document served, transmitted and rendered. </li>
<li>no concept of a "connection" at HTTP-protocol level, <br />so authentication and security related stuff (e.g. SSL handshake) has to be repeated with each request/response (=change of GUI state). </li>
</ul>
<br />
This allowed WebGUIs to max out on bandwidth <b>in</b>efficiency and latency. It also leads to usability nuggets by losing subtle GUI state (such as scrollbar position or the longish forum post you started to write).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2Z2SlsQwVlrLLDXLFhqWd7fzhavCzby1y9Qh4jl54PkH1aP8hddvjnc9oGuwOqUgJbXLs6KJQByZdumgW8vpUPRz_pXzxVvXy23nqMYnZN6a_pI02tX7aJVtQs0sEGISMKYc0gYYfb7s/s1600/web99.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2Z2SlsQwVlrLLDXLFhqWd7fzhavCzby1y9Qh4jl54PkH1aP8hddvjnc9oGuwOqUgJbXLs6KJQByZdumgW8vpUPRz_pXzxVvXy23nqMYnZN6a_pI02tX7aJVtQs0sEGISMKYc0gYYfb7s/s640/web99.PNG" height="278" width="640" /></a></div>
<br />
<h2>
JavaScript, Ajax and "Look ma, a menu in the browser"</h2>
Innovation (& browser wars) never sleep. Industry leading companies and consortium's figured out, that the best way to address fundamental design issues is to provide many many many workarounds on top of it.<br />
<br />
Examples:<br />
<ul>
<li>Http Keep alive to reduce latency</li>
<li>Standardize on an untyped weirdish scripting language, <br />which can be cluttered all across the html page in tiny snippets. Ah .. there is nothing like mixing program, content and styling into a big code soup ..</li>
<li>Content Caching (man we had fun doing software updates ..) <br />to avoid superfluous reload of static content</li>
<li>Define new Html Tags. A lot of them. <br />The advantage of having many html tags is, that browsers have more room to implement them differently, allowing for the popular "if ( ie6 ) {...........} else if (mozilla) .. else if (safari) .." coding style. This way bandwidth requirements could be extended even more.</li>
<li>CSS was embraced, <br />so when done with definition of new tags, they could continue with definition of new css attributes. Also javascript could mess up even better modifying the HTML DOM and CSS definitions at runtime.</li>
<li>Async Http Request (AJAX) <br />avoids the need to reload whole html-page.</li>
</ul>
These are kludges (some of them with notable negative side effects) addressing fundamental issues of an overly simplistic foundation technology. So we got, what I call the MESS technology stack:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLugG_gSJhtzlcypPetAThHV5c7Yi5AbnG1NfgmovAURCo9JZHvRxjlf8JAlRg3Torx7BQVFMLsgvGA63iVSOra2YTi-6lEg5ASTAGNVxZWN7YDRXMbGJb2_ZbJJRgZc-QCN7KomL3zNI/s1600/web03.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLugG_gSJhtzlcypPetAThHV5c7Yi5AbnG1NfgmovAURCo9JZHvRxjlf8JAlRg3Torx7BQVFMLsgvGA63iVSOra2YTi-6lEg5ASTAGNVxZWN7YDRXMbGJb2_ZbJJRgZc-QCN7KomL3zNI/s1600/web03.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
The MESS principles also infiltrated server side frameworks, as they had to deal with the MESS somehow. I'd speculate some of the frameworks are 90% "mess-handlers', designed to virtualize a solid engineered foundation in order to increase productivity when messing with the MESS.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h2>
Google to the rescue !!</h2>
<div>
One can safely assume apple and microsoft are not *that* interested in providing a fluid web application platform. If all applications would be WebApplications, the importance of the underlying operation system dwindles. All a user needs, is a compliant browser (hello Chrome OS).</div>
<div>
<br /></div>
<div>
Google as a major web application implementor/provider suffer them self from the high.cost@low-quality.webapp syndrome.</div>
<div>
<br />
Some clever JavaScript hackers (<= positive) started to figure out a simplified, more flexible web application design. They basically implemented a "Fat Client" in JavaScript. Instead of doing page-reloading Http requests, they used Async Http to retrieve data from the server and rendered this data by modifying parts of the displayed html document (via DOM). Whenever you see a responsive 'cool' web application, you can safely assume it relies on this approach.<br />
<br />
However JavaScript is not designed to work well at large business applications with many lines of code and fluctuation of development staff as typical for enterprise software development. It becomes a maintenance nightmare for sure (<b>if </b>the project succeeds at all) or a minimalistic solution hated by end-users.<br />
<br /></div>
<div>
The first widely known step in backing up the hype with a structured technology was the Google Widget Toolkit, which relied on compiling ("transpiling") java to browser compatible JavaScript.<br />
For various reasons, this project has been stalled (but not dropped) by Google in favour of their new Dart language before the technology reached a 'mature' state. It has been reported that turn-around times get unacceptable with larger applications and its unlikely this will change (because resources have been moved to Dart). Remembering the law suit of Oracle against use of Java in Android, this move of Google is not too surprising.<br />
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif_YmIGq-YnozKFzKGVLXmxHD-dxjaax7l9XCrOTsm8afKzX6XibzmCXMX9afftjQe-kBW5XacA3m6B0jDXwNsR4FRyovHD6JwaUJljQGtQn8O9L73mDQ4Wu7621LLPqbofDGDYSsrRCU/s1600/webfuture1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif_YmIGq-YnozKFzKGVLXmxHD-dxjaax7l9XCrOTsm8afKzX6XibzmCXMX9afftjQe-kBW5XacA3m6B0jDXwNsR4FRyovHD6JwaUJljQGtQn8O9L73mDQ4Wu7621LLPqbofDGDYSsrRCU/s1600/webfuture1.PNG" /></a></div>
Dart has the potential to reach a level of convenience which has been there for native apps for years:<br />
<br />
<ul>
<li>extensive API to the browser. Anything achievable with JavaScript can be done with Dart.</li>
<li>a modern structured language with (optional) compile time type checking, a package concept and support for many well-known principles of object oriented programming like Classes, Interfaces (Mixins), Inheritance, Lambdas. The concurrency model of Dart (Isolates, similar to Actors) could be interesting on the server side also.</li>
<li>compilable to JavaScript</li>
<li>support for fully encapsulated tag-based UI components (via polymer lib and upcoming WebComponents w3c standard)</li>
<li>backed by a company providing the worlds leading browser and a believable self-dependency on the technology.</li>
</ul>
<br />
<h2>
Conclusion</h2>
<div>
Mankind not only is capable to build chips with 800 million transistors, it also might be capable creating standards defined by 800 million words .. scary ;-) .</div>
<div>
<br /></div>
<div>
<h2>
What about Java ?</h2>
</div>
<div>
Update: Back2Brwsr and TeaVM are projects based on Java=>JS transpiling.<br />
<br />
While the Scala guys join the party late <a href="http://www.scala-lang.org/news/2013/11/29/announcing-scala-js-v0.1.html">http://www.scala-lang.org/news/2013/11/29/announcing-scala-js-v0.1.html</a>, Oracle misses the train and seems to favour JavaFX as a GUI platform (my prediction: Will fall asleep prior to reaching maturity).</div>
<div>
GWT has been open sourced, so maybe the community will do further development. Given that there are more open source projects than developers willing to contribute, I doubt this will be able to keep pace with Google's money backed Dart. Also I could imagine the (aging) Java community shys away from (partially) deprecating their habitual server side framework landscape.</div>
<div>
<br /></div>
<div>
There is also <a href="http://qooxdoo.org/contrib/project/qwt/about/java">http://qooxdoo.org/contrib/project/qwt/about/java</a> backed by a bigger german hosting company. I have not tested this, but I doubt that pure community driven development will keep pace. Being compliant to ever changing draft specs and dealing with browser implementation variance is not fun ...</div>
<div>
<br /></div>
<div>
What's required is a byte-code to JavaScript transpiler. </div>
<div>
The browser API (DOM access) could be modeled by a set of interfaces at compile time and replaced by the transpiler with the appropriate javascript calls. </div>
<div>
Using static bytecode analysis or by tracking required classes at runtime, the set of required transpiled classes/methods could be found. Additionally one would need a fall-back transpiler at runtime. If a client side script instantiates an "un-transpiled" java class or method, it could be transpiled on-the-fly at server side and loaded to the client.</div>
<div>
Nice thing of such a bytecode level transpiler would be, that all VM-based languages (groovy, scala, jruby, jphyton, ..) could profit from such a system. </div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioiKP_qdTSFvwitcJoLh7IlS_5eJ-BI7Si5GKN_AZ9Oy5Rgjb_DDiQIu8pnO76gch1hbUJL3MOVT0JS1qFWuDem9IKgZREdlv1K6BQ-OfGxtsugekYwoBk9Bf7Z7ZQhA3hBZafjEWX0iU/s1600/javatrans.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioiKP_qdTSFvwitcJoLh7IlS_5eJ-BI7Si5GKN_AZ9Oy5Rgjb_DDiQIu8pnO76gch1hbUJL3MOVT0JS1qFWuDem9IKgZREdlv1K6BQ-OfGxtsugekYwoBk9Bf7Z7ZQhA3hBZafjEWX0iU/s640/javatrans.PNG" height="278" width="640" /></a></div>
<i>Example of a possible Java2JavaScript transpiling solution: Since Java is a wide spread server platform, a java2js transpiler could work dynamically, which is a major strategic advantage over Dart, which needs to use static analysis & compilation because most server systems are not running native dart.</i><br />
<br /></div>
<div>
There needs to be put real money on such a Java solution, with slow moving JCP we probably can expect something like this 2016 or later ...<br />
Would be fun building such a transpiler system .. but no time - gotta mess with the MESS.<br />
<br />
<b>Update</b>: The J2EE java 8 project seems to adapt to new 'thin' webapp architecture with project "<b>Avatar</b>". I still haven't figured out the details. On a first glance it looks like they bring JS to the server instead of bringing Java transpilation to the client .. which is a mad idea imho :-).<br />
<br />
<b>Update on GWT: </b>A google engineer answered back on GWT telling me the project is alive and is in active development. It turns out Dart and GWT are handled by distinct (competing?) divisions of google. There still is evidence, that Dart receives the real money while GWT is handed over to the community. Google plans to bring native DartVM to chrome within ~1 year (rumours) and plans on delivering the DartVM to Android also.</div>
<div>
<br /></div>
<div>
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com31tag:blogger.com,1999:blog-4356402223581768063.post-13737424586575489892013-10-12T00:24:00.000+02:002014-03-07T14:53:28.636+01:00Still using Externalizable to get faster Serialization with Java ?<br />
<b>Update: </b>When turning off detection of cycles, performance can be improved even further (FSTConfiguration.setUnshared(true)). However in this mode an object referenced twice is written twice.<br />
<br />
According to popular belief, the only way to go is handcrafted implementation of the Externalizable interface in order to get fast Java Object serialization.<br />
<br />
Manual coding of "readExternal" and "writeExternal" methods is an errorprone and boring task. Additionally, with each change of a class's fields, the externalizable methods need adaption.<br />
<br />
In contradiction to popular belief, a good implementation of generic serialization can be faster than handcrafted implementation of the Externalizable interface 99% if not 100% of the time.<br />
<br />
Fortunately I managed to <b>save the world</b> with the <b><a href="https://github.com/RuedigerMoeller/fast-serialization" target="_blank">fast-serialization library</a></b> by addressing the disadvantages of Serialization vs Externalizable.<br />
<br />
<b>The Benchmark</b><br />
<br />
The following class will be benchmarked.<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;">public class BlogBench implements Serializable {</span><br />
<span style="font-size: x-small;"> public BlogBench(int index) {</span><br />
<span style="font-size: x-small;"> // avoid benchmarking identity references instead of StringPerf</span><br />
<span style="font-size: x-small;"> str = "Some Value "+index;</span><br />
<span style="font-size: x-small;"> str1 = "Very Other Value "+index;</span><br />
<span style="font-size: x-small;"> switch (index%3) {</span><br />
<span style="font-size: x-small;"> case 0: str2 = "Default Value"; break;</span><br />
<span style="font-size: x-small;"> case 1: str2 = "Other Default Value"; break;</span><br />
<span style="font-size: x-small;"> case 2: str2 = "Non-Default Value "+index; break;</span><br />
<span style="font-size: x-small;"> }</span><br />
<span style="font-size: x-small;"> }</span><br />
<span style="font-size: x-small;"> private String str;</span><br />
<span style="font-size: x-small;"> private String str1;</span><br />
<span style="font-size: x-small;"> private String str2;</span><br />
<span style="font-size: x-small;"> private boolean b0 = true;</span><br />
<span style="font-size: x-small;"> private boolean b1 = false;</span><br />
<span style="font-size: x-small;"> private boolean b2 = true;</span><br />
<span style="font-size: x-small;"> private int test1 = 123456;</span><br />
<span style="font-size: x-small;"> private int test2 = 234234;</span><br />
<span style="font-size: x-small;"> private int test3 = 456456;</span><br />
<span style="font-size: x-small;"> private int test4 = -234234344;</span><br />
<span style="font-size: x-small;"> private int test5 = -1;</span><br />
<span style="font-size: x-small;"> private int test6 = 0;</span><br />
<span style="font-size: x-small;"> private long l1 = -38457359987788345l;</span><br />
<span style="font-size: x-small;"> private long l2 = 0l;</span><br />
<span style="font-size: x-small;"> private double d = 122.33;</span><br />
<span style="font-size: x-small;">}</span></blockquote>
To implement Externalizable, a copy of the class above is made but with Externalizable implementation<br />
(source is <a href="http://code.google.com/p/fast-serialization/source/browse/trunk/src/test/java/de/ruedigermoeller/serialization/testclasses/blog/BlogBenchExternalizable.java" target="_blank">here</a>).<br />
The main loop for fast-serialization is identical, I just replace "ObjectOutputStream" with "FSTObjectOutput" and "ObjectInputStream" with "FSTObjectInput".<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7whAMAV488zg3im1s1dSQ7JgOU_26psqRwZEuMJz-ICinUgECIo6YLwI5A0VhqsqTaMDX5pg8-YpiG5OP92cgRI1kcVP7Hms-4_XpS3Rcw_jvh0fZjftaZn2tscO_ZbqrUF3doZsFFZY/s1600/dropin-perf.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7whAMAV488zg3im1s1dSQ7JgOU_26psqRwZEuMJz-ICinUgECIo6YLwI5A0VhqsqTaMDX5pg8-YpiG5OP92cgRI1kcVP7Hms-4_XpS3Rcw_jvh0fZjftaZn2tscO_ZbqrUF3doZsFFZY/s640/dropin-perf.PNG" height="240" width="640" /></a></div>
<b>Result</b>:<br />
<br />
<ul>
<li>Externalizable performance with JDK-Serialization is much better compared to Serializable</li>
<li>FST manages to serialize faster than manually written "read/writeExternal' implementation</li>
</ul>
<br />
Size is 319 bytes for JDK Serializable, 205 for JDK Externalizable, 160 for FST Serialization. Pretty big gain for a search/replace operation vs handcrafted coding ;-). BTW if the "Externalizable" class is serialized with FST it is still slightly slower than letting FST do generic serialization.<br />
<br />
<b>There is still room for improvement ..</b><br />
<br />
The test class is rather small, so setup + allocation of the Input and Output streams take a significant part on the times measured. Fortunately FST provides mechanisms to reuse both FSTObjectInput and FSTObjectOutput. This yields ~200ns better read and write times.<br />
<br />
So "new FSTObjectOutput(inputStream)" is replaced with<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;">FSTConfiguration fstConf = FSTConfiguration.getDefaultConfiguration();<br />...<br />fstConf.getObjectOutput(bout)</span></blockquote>
<b>There is even more improvement ..</b><br />
<br />
Since Externalizable does not need to track references and this is not required for the test class, we turn off reference tracking for our sample by using the @Flat annotation. We can also make use of the fact, "str3" is most likely to contain a default value ..<br />
<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><b>@Flat</b><br />public class BlogBenchAnnotated implements Serializable {<br /> public BlogBenchAnnotated(int index) {<br /> // avoid benchmarking identity references instead of StringPerf<br /> str = "Some Value "+index;<br /> str1 = "Very Other Value "+index;<br /> switch (index%3) {<br /> case 0: str2 = "Default Value"; break;<br /> case 1: str2 = "Other Default Value"; break;<br /> case 2: str2 = "Non-Default Value "+index; break;<br /> }<br /> }<br /> <b>@Flat</b> private String str;<br /> <b>@Flat</b> private String str1;<br /> <b>@OneOf</b>({"Default Value","Other Default Value"})<br /> <b>@Flat</b> private String str2;</span></blockquote>
<br />
<br />
<b>and another one ..</b><br />
<b><br /></b>
To be able to instantiate the correct class at readtime, the classname must be transmitted. However in many cases both reader and writer know (at least most of) serialized classes at compile time. FST provides the possibility to register classes in advance, so only a number instead of a full classname is transmitted.<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;">FSTConfiguration.getDefaultConfiguration().registerClass(BlogBenchAnnotated.class);</span></blockquote>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvsDVHKdnCCfzpk7QShLtFr_BqfHAOMPtlqreGoziHGuYlUlhAUOu8fr4eYKZWFBClD5ZNXzIk0XYi1bVZ0u0uz_Zgj4AJd3OfNqY00AhJpSewDiVXzL4paVMmGv2ZqDeH41XRjN5WrHg/s1600/fstopt.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvsDVHKdnCCfzpk7QShLtFr_BqfHAOMPtlqreGoziHGuYlUlhAUOu8fr4eYKZWFBClD5ZNXzIk0XYi1bVZ0u0uz_Zgj4AJd3OfNqY00AhJpSewDiVXzL4paVMmGv2ZqDeH41XRjN5WrHg/s1600/fstopt.PNG" /></a></div>
<br />
<b>What's "Bulk"</b><br />
<b><br /></b>
Setup/Reuse of Streams actually require some nanoseconds, so by benchmarking just read/write of a tiny objects, a good part of per-object time is stream init. If an array of 10 BenchMark objects is written, per object time goes <300ns per object read/write.<br />
Frequently an application will write more than one object into a single stream. For RPC encoding applications, a kind of "Batching" or just writing into the same stream calling "flush" after each object are able to actually get <300ns times in the real world. Of course Object Reference sharing must be turned off then (FSTConfiguration.setShared(false)).<br />
<br />
For completeness: JDK (with manual Externalizable) Bulk yields 1197 nanos read and 378 nanos write, so it also profits from less initilaization. Unfortunately reuse of ObjectInput/OutputStream is not that easy to achieve mainly because ObjectOutputStream already writes some bytes into the underlying stream as it is instantiated.<br />
<br />
Note that if (constant) initialization time is taken out of the benchmarks, the relative performance gains of FST are even higher (see benchmarks on fast serialization site).<br />
<br />
Links:<br />
<br />
<a href="http://code.google.com/p/fast-serialization/source/browse/#svn%2Ftrunk%2Fsrc%2Ftest%2Fjava%2Fde%2Fruedigermoeller%2Fserialization%2Ftestclasses%2Fblog%253Fstate%253Dclosed" target="_blank">Source of this Benchmark</a><br />
<a href="https://github.com/RuedigerMoeller/fast-serialization" target="_blank">FST Serialization Library</a> (moved to github from gcode recently)<br />
<br />
<br />
<br />Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com74tag:blogger.com,1999:blog-4356402223581768063.post-45284516428161784582013-07-31T15:49:00.001+02:002013-07-31T20:16:53.609+02:00Impact of large primitive arrays (BLOBS) on Garbage CollectionWhile OldGen GC duration ("FullGC") depends on number of objects and the locality of their references to each other, young generation duration depends on the size of OldSpace memory. Alexej Ragozin has made extensive tests which give a good impression of young GC duration vs HeapSize <a href="http://blog.ragozin.info/2013/06/java-gc-in-numbers-parallel-young.html" target="_blank">here</a>.<br />
<br />
In order to avoid heavy impact of long lived data on OldGen GC, there are several workarounds/techniques.<br />
<br />
<ol>
<li>Put large parts of mostly static data Off-Heap using ByteBuffer.allocateDirect() or Unsafe.allocateMemory(). This memory is then used to store data (e.g. by using a fast serialization like <a href="http://code.google.com/p/fast-serialization/">http://code.google.com/p/fast-serialization/</a> [oops, I did it again] or specialized solutions like <a href="http://code.google.com/p/vanilla-java/wiki/HugeCollections">http://code.google.com/p/vanilla-java/wiki/HugeCollections</a> ).<br />Downside is, that one frequently has to implement a manual memory mangement on top.<br /></li>
<li>"instance saving" on heap by serializing into byte-arrays or transformation of datastructures. This usually involves using open adressed hashmaps without "Entry" Objects, large primitive arrays instead of small Objects like<br /><span style="font-family: Courier New, Courier, monospace;">class ReferenceDataArray {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> int x[];</span><br />
<span style="font-family: Courier New, Courier, monospace;"> double y[];</span><br />
<span style="font-family: Courier New, Courier, monospace;"> long z[];</span><span style="font-family: 'Courier New', Courier, monospace;"> </span><br />
<span style="font-family: Courier New, Courier, monospace;"> public ReferenceDataArray(int size) {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> x = new int[size];</span><br />
<span style="font-family: Courier New, Courier, monospace;"> y = new ...;</span><br />
<span style="font-family: Courier New, Courier, monospace;"> z = ...;</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public long getZ(int index) { </span><span style="font-family: 'Courier New', Courier, monospace;">return z[index]; </span><span style="font-family: 'Courier New', Courier, monospace;">}</span><br />
<span style="font-family: Courier New, Courier, monospace;">}</span>,<br />
replacement of generic collections with <Integer>, <Long> by specialized implementations with direct primitve int, long, ..<br />If its worth to cripple your code this way is questionable, however the option exists.</li>
</ol>
<br />
Going the route outlined in (2) improves the effectivity of OldGen GC a lot. FullGC duration can be in the range of 2s even with heap sizes in the 8 GB area. CMS performs significantly better as it can scan OldSpace faster and therefore needs less headroom in order to avoid Full GC.<br />
<br />
However there is still the fact, that YoungGen GC scales with OldSpace size.<br />
<br />
The scaling effect is usually associated with "cardmarking". Young GC has to remember which areas of OldSpace have been modified (in such a way they reference objects in YoungGen). This is done with kind of a BitField where each bit (or byte) denotes the state of (modified/reference created or similar) a chunk ("card") of OldSpace.<br />
Primitive Arrays basically are BLOBS for the VM, they cannot contain a reference to other Java Objects, so theoretically there is no need to scan or card-mark areas containing BLOBS them when doing GC. One could think e.g. of allocating large primitive arrays from top of oldspace, other objects from bottom this way reducing the amount of scanned cards.<br />
<br />
<b>Theory: blobs (primitive arrays) result in shorter young GC pauses then equal amount of heap allocated in smallish Objects.</b><br />
<br />
Therefore I'd like to do a small test, measuring the effects of allocating large primitive arrays (such as byte[], int[], long[], double[]) on NewGen GC duration.<br />
<br />
<h3>
The Test</h3>
<blockquote class="tr_bq">
public class BlobTest {<br />
static ArrayList blobs = new ArrayList();<br />
static Object randomStuff[] = new Object[300000];<br />
public static void main( String arg[] ) {<br />
if ( Runtime.getRuntime().maxMemory() > 2*1024*1024*1024l) { // 'autodetect' avaiable blob space from mem settings<br />
int blobGB = (int) (Runtime.getRuntime().maxMemory()/(1024*1024*1024l));<br />
System.out.println("Allocating "+blobGB*32+" 32Mb blobs ... (="+blobGB+"Gb) ");<br />
for (int i = 0; i < blobGB*32; i++) {<br />
blobs.add(new byte[32*1024*1024]);<br />
}<br />
System.gc(); // force VM to adapt ..<br />
}<br />
// create eden collected tmps with a medium promotion rate (promotion rate can be adjusted by size of randomStuff[])<br />
while( true ) {<br />
randomStuff[((int) (Math.random() * randomStuff.length))] = new Rectangle();<br />
}<br />
}<br />
}</blockquote>
<br />
<br />
The while loop at the bottom simulates the allocating application. Because I rewrite random indizes of the randomStuff arrays using a random index, a lot of temporary objects are created, because they same index is rewritten with another object instance. However because of random, some indices will not be hit in time and live longer, so they get promoted. The larger the array, the less likely index overwriting gets, the higher the promotion rate to OldSpace.<br />
<br />
In order to avoid bias by VM-autoadjusting, I pin NewGen sizes, so the only variation is the allocation of large byte[] on top the allocation loop. (Note these settings are designed to encourage promotion, they are in now way optimal).<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
commandline:<br />
<br />
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
java <b>-Xms1g -Xmx1g</b> -verbose:gc -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=12 -XX:NewSize=100m -XX:MaxNewSize=100m -XX:MaxTenuringThreshold=2</div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
by adding more GB the upper part of the test will use any heap above 1 GB to allocate byte[] arrays.</div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
<br />
java <b>-Xms3g -Xmx3g</b> -verbose:gc -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=12 -XX:NewSize=100m -XX:MaxNewSize=100m -XX:MaxTenuringThreshold=2</div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
...</div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
...</div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
<div style="background-color: white; border: 0px; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px; margin: 0px; padding: 0px; vertical-align: baseline;">
<span style="background-color: transparent;">java</span><span style="background-color: transparent;"> </span><b>-Xms11g -Xmx11g</b> -verbose:gc -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=12 -XX:NewSize=100m -XX:MaxNewSize=100m -XX:MaxTenuringThreshold=2</div>
<div>
<br /></div>
<div>
I am using byte[] arrays in the test, I verified int[], long[] behave exactly the same (must apply divisor then to adjust for larger size).</div>
</div>
<br />
<br />
<h3>
Results</h3>
<div>
(jdk 1.7_u21)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitXUSsbYGoXtoXDVZ7AJOsgighaiEimcKZfuCrVm7jRPkTH-CFQq7IXTRd69XAaBS7GkYOF4afv-BDAAKEYwwkTaL-LuY2JAdqFcpqbxgQj3aRyDgKXZCS5tnK305OPVLxlEr4T2yi8gk/s1600/defyoungblobscal.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitXUSsbYGoXtoXDVZ7AJOsgighaiEimcKZfuCrVm7jRPkTH-CFQq7IXTRd69XAaBS7GkYOF4afv-BDAAKEYwwkTaL-LuY2JAdqFcpqbxgQj3aRyDgKXZCS5tnK305OPVLxlEr4T2yi8gk/s400/defyoungblobscal.PNG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLGim4A1Jsd7ps1cJJU974YFOImPTifkSqZWcj6AbMWn_RtJxB35JBu4kqasWd9yal1SUQ0RU0S5sW3sxsO9w2l36fAhE2gUiP-tJkAHeJO4AmIq8GKOnVyrBKSkcRlDmZlSfPXBDr4Lk/s1600/defblobsscal.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLGim4A1Jsd7ps1cJJU974YFOImPTifkSqZWcj6AbMWn_RtJxB35JBu4kqasWd9yal1SUQ0RU0S5sW3sxsO9w2l36fAhE2gUiP-tJkAHeJO4AmIq8GKOnVyrBKSkcRlDmZlSfPXBDr4Lk/s400/defblobsscal.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgg7V8-fsg7voghDFO4t7x-aNChAOkCMUOkhJ8QSuv0OqeYHB4cBUMAd8s8BjmUWfxoHjtXJY9Z6MB2ZsIGwGvsahXk6Urx0y5cdGZD7vzr80LOgC4scZzUQ8IOK0LROtULnxxzSsydkc/s1600/cmsblobscal.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgg7V8-fsg7voghDFO4t7x-aNChAOkCMUOkhJ8QSuv0OqeYHB4cBUMAd8s8BjmUWfxoHjtXJY9Z6MB2ZsIGwGvsahXk6Urx0y5cdGZD7vzr80LOgC4scZzUQ8IOK0LROtULnxxzSsydkc/s400/cmsblobscal.PNG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi55Qqv15eCH8DAAHp2bG_XW2Qy7GS4rXdQDAVT6cz1J58bgj0wyGVNXsZtfBNIxXfNQZlVT94pXEvf444EWAjQbSPIn5hAKMEMGe32L5uQULYrBDTI7PBnZSlvW1xb9aGSp7Ff8iW8hB4/s1600/ogenblobs.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi55Qqv15eCH8DAAHp2bG_XW2Qy7GS4rXdQDAVT6cz1J58bgj0wyGVNXsZtfBNIxXfNQZlVT94pXEvf444EWAjQbSPIn5hAKMEMGe32L5uQULYrBDTI7PBnZSlvW1xb9aGSp7Ff8iW8hB4/s320/ogenblobs.PNG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjyhOKPcF2OMe0cX3XGoFSjRYfzUb8BI6SEsdN1ECVCLpuk_QKtja7fwEj071dq6VxdXPBoRDlr7CISPDxcMKzERduCGwQw2TDrOkmG9PxlYznSut1njR9xCj5oVwxv7BGjybQ2Z3jgIM/s1600/ygenblobs.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjyhOKPcF2OMe0cX3XGoFSjRYfzUb8BI6SEsdN1ECVCLpuk_QKtja7fwEj071dq6VxdXPBoRDlr7CISPDxcMKzERduCGwQw2TDrOkmG9PxlYznSut1njR9xCj5oVwxv7BGjybQ2Z3jgIM/s400/ygenblobs.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The 'Objects' test was done by replacing the static byte[] allocation loop in the benchmark by</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for ( int i = 0; i < blobGB*2700000; i++ )</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> nonblobs.add(new Object[] {<br /> new Rectangle(),new Rectangle(),new Rectangle(),<br /> new Rectangle(),new Rectangle(),new Rectangle(),<br /> new Rectangle(),new Rectangle(),new Rectangle(),<br /> new Rectangle()});</span></div>
<div>
<br /></div>
<br />
<h3>
Conclusion</h3>
</div>
<blockquote class="tr_bq">
Flattening data structures using on-heap allocated primitive arrays ('BLOBS') reduces OldGen GC overhead very effective. </blockquote>
<blockquote class="tr_bq">
Young Gen pauses slightly reduce for CMS, so scaling with OldGen size is damped but not gone. For DefaultGC (PSYoung), minor pauses are actually slightly higher when the heap is filled with BLOBs.</blockquote>
<blockquote class="tr_bq">
I am not sure if the observed young gen duration variance has anything to do with "card marking" however i am satisfied quantifying effects of different allocation types and sizes :-) </blockquote>
<br />
<h3>
Further Improvement incoming ..</h3>
<div>
<br /></div>
<div>
With this genious little optimization coming up in JDK 7_u40</div>
<div>
<br /></div>
<div>
<a href="http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068625" style="background-color: white; border: 0px; color: #6611cc; cursor: pointer; font-family: Arial, Helvetica, sans-serif; font-size: 13px; margin: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;" target="_blank">http://bugs.sun.com/<wbr></wbr>bugdatabase/view_bug.do?bug_<wbr></wbr>id=7068625</a></div>
<div>
<br /></div>
<div>
card scanning of unmarked cards speeds up by a factor of 8. </div>
<div>
<br />
Additonally notice </div>
<div>
<a href="http://blog.ragozin.info/2012/03/secret-hotspot-option-improving-gc.html">http://blog.ragozin.info/2012/03/secret-hotspot-option-improving-gc.html</a></div>
<div>
<br /></div>
(for the test <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-XX:+UnlockDiagnosticVMOptions -XX:ParGCCardsPerStrideChunk=512</span> gave the best results)<br />
At least CMS Young Gen pause scaling is not too bad.<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdrncqIqKZ5gB_RftE2-sQZLjcun510GsVWlczQuNeSuot6RcxZFMI7g6go-fU4buplvZGesDe8WXg3To5gYGrkO8iGhViNclgDJmaFoe0BPuE68OEsYuklk3O_2hpG4qdD5-Ac1_gUn0/s1600/7u40.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdrncqIqKZ5gB_RftE2-sQZLjcun510GsVWlczQuNeSuot6RcxZFMI7g6go-fU4buplvZGesDe8WXg3To5gYGrkO8iGhViNclgDJmaFoe0BPuE68OEsYuklk3O_2hpG4qdD5-Ac1_gUn0/s400/7u40.PNG" width="400" /></a></div>
<h3>
And G1 ?</h3>
<div>
G1 fails to execute the test. If one only allocates 6GB of byte[] with 11GB of heap, it still is much more disruptive than CMS. It works if I use small byte[] chunks of 1MB size and set page size to 32MB. Even then pauses are longer compared to CMS. G1 seems to have problems with large object arrays which will be problematic for IO intensive applications requiring big and many byte buffers.</div>
<div>
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com134tag:blogger.com,1999:blog-4356402223581768063.post-6913246861218661612013-07-28T20:42:00.002+02:002013-07-29T04:50:14.543+02:00What drives Full GC duration ?<h2>
</h2>
<div>
Its obvious that, the more memory allocated, the longer a full GC pause will be. However a GC has to track references, so I want to find out if primitve fields (like int, long) also increase Full GC duration. Additionally I'd like to find if the structure of an object graph has impact on GC duration.</div>
<div>
<br /></div>
<h2>
The Test</h2>
<div>
I am creating a number of instances of the following classes:</div>
<div>
<blockquote class="tr_bq">
static class GCWrapper {<br />
GCWrapper(Object referenced) {<br />
this.referenced = referenced;<br />
}<br />
Object referenced;<br />
}</blockquote>
</div>
<div>
<blockquote>
static class GCWrapperMiddle {<br />
int a,b,c,d,e,f,g,h,i,j,k,l;<br />
GCWrapperMiddle(Object referenced) {<br />
this.referenced = referenced;<br />
}<br />
Object referenced;<br />
} </blockquote>
<blockquote>
static class GCWrapperFat {<br />
int a,b,c,d,e,f,g,h,i,j,k,l;<br />
long aa,ab,ac,ad,ae,af,ag,ah,ai,jj,ak,al;<br />
GCWrapperFat(Object referenced) {<br />
this.referenced = referenced;<br />
}<br />
Object referenced;<br />
}</blockquote>
</div>
<div>
they all hold exactly one reference. But "Middle" and "Fat" additionally contain primitive fields, which increases their memory consumption a lot.<br />
I am chaining those objects after creation to a linked list. For locality tests, the reference is pointed to a random other "GCWrapper" Object to create "far" references.</div>
<div>
Additionally I am using the class below to test effects of "many references", a highly interlinked Object graph:</div>
<div>
<blockquote class="tr_bq">
static class GCWrapperLinked {<br />
GCWrapperLinked(Object referenced) {<br />
this.r1 = referenced;<br />
}<br />
Object r1,r2,r3,r4,r5;<br />
}</blockquote>
</div>
<div>
After instantiation, the test does 5 GC's using System.gc() and takes the last value (~3 calls are required until time stabilizes). </div>
<div>
Tests are done on an Intel i7 4 core 8 threads mobile processor with a somewhat outdated JDK 7 u 21, as I am in vacation currently with a pretty bad internet connection ...</div>
<div>
<br />
<h2>
What has more impact, Memory Size or Number of Object (-References) ?</h2>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij3WZmNwRFw1-wly6z691g2sZ-MCJOvhlsCfAniwhHu-amv85tmqRdVREqhy1d5GtOlhAQZaAtazsIA-bjB4NuAptWu-tT8CbBJ6EVjDPNdT7ua9EdOEtyWEakMbvVWC-MkwepNTlYWZ4/s1600/numobj_real.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="336" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij3WZmNwRFw1-wly6z691g2sZ-MCJOvhlsCfAniwhHu-amv85tmqRdVREqhy1d5GtOlhAQZaAtazsIA-bjB4NuAptWu-tT8CbBJ6EVjDPNdT7ua9EdOEtyWEakMbvVWC-MkwepNTlYWZ4/s640/numobj_real.PNG" width="640" /></a></div>
<div>
The test was done with all collectors and different number of Objects on the Heap. Objects were allocated in a linked List like<br />
<blockquote class="tr_bq">
GCWrapperMiddle res = new GCWrapperMiddle(null);<br />
for ( int i = 0; i < len; i++ ) {<br />
res = new GCWrapperMiddle(res);<br />
}<br />
return res;</blockquote>
<div>
<br /></div>
The result shows clear linear dependency inbetween the number of Objects and Full GC duration.<br />
Please don't conclude Default GC (ParOldGC) is slowest from the results, because<br />
a) subsequent calls to system.gc() without mutation may enable Collectors to cut corners</div>
<div>
b) they might choose to use fewer/more threads during collection<br />
<br />
I test all Collectors just to ensure that observed interdependencies are present for all collectors.<br />
<br />
Now let's compare effects of Object size on Full GC durations. As described above, I pumped up object size by adding int and long fields to them. Those fields cannot contain references to other objects (no need to scan them) but still consume space. The test creates 6 million objects of the classes shown above. Heap consumption is:<br />
<br />
<ul>
<li>163 Mb for 'small'</li>
<li>427 Mb for 'middle'</li>
<li>1003 Mb for 'large'</li>
</ul>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXJNhSjVFBqKagFpdmer6krhy5X2jtz515sQQorNWSj9YSvLP7ldTrikBo5mdIQ5xF7dUDwVXmzep9ulhpY3NZY0qDHHS9t_YTD_L52vpdJDJFiXwK6EcVZE-5ft8f5g0RTdxRcfMAYFk/s1600/numobj.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXJNhSjVFBqKagFpdmer6krhy5X2jtz515sQQorNWSj9YSvLP7ldTrikBo5mdIQ5xF7dUDwVXmzep9ulhpY3NZY0qDHHS9t_YTD_L52vpdJDJFiXwK6EcVZE-5ft8f5g0RTdxRcfMAYFk/s640/numobj.PNG" width="640" /></a></div>
<div>
Now that's interesting: <br />
<b>Object size has a minor impact on Full GC duration (the differences observed can be attributed mostly to CPU cache misses, i assume). Main driver of FullGC duration is the number of </b><b>object (-references). </b></div>
<div>
<br />
<br />
<h2>
Does GC duration depend on number of objects or the number of object-references ?</h2>
</div>
<div>
<br />
To test this, I create an object graph where each gcwrapper object points to another randomly choosen gcwrapper object.</div>
<div>
<blockquote class="tr_bq">
GCWrapper res[] = new GCWrapper[len];<br />
for ( int i = 0; i < len; i++ ) {<br />
res[i] = new GCWrapper(null);<br />
}<br />
for ( int i = 0; i < len; i++ ) {<br />
res[i].referenced = res[((int) (Math.random() * len))];<br />
}<br />
return res;</blockquote>
</div>
<div>
and ('5 refs')</div>
<div>
<blockquote class="tr_bq">
GCWrapperLinked res[] = new GCWrapperLinked[len];<br />
for (int i = 0; i < res.length; i++) {<br />
res[i] = new GCWrapperLinked(null);<br />
}<br />
for (int i = 0; i < res.length; i++) {<br />
res[i].r1 = res[((int) (Math.random() * len))];<br />
res[i].r2 = res[((int) (Math.random() * len))];<br />
res[i].r3 = res[((int) (Math.random() * len))];<br />
res[i].r4 = res[((int) (Math.random() * len))];<br />
res[i].r5 = res[((int) (Math.random() * len))];<br />
}<br />
return res;</blockquote>
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIpnaeUKnol9np04YfleJTClZiI4xnX8n3nnXEsE1xfIJVK0vHh6FqkecLapgodwT0NZV4v4fER9-_HgefGLdvup2izeQFSFdNF60Mx8nkHlA-hNt_qy78YIW3uSyVuE41im79VjnJrpg/s1600/linkage.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="350" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIpnaeUKnol9np04YfleJTClZiI4xnX8n3nnXEsE1xfIJVK0vHh6FqkecLapgodwT0NZV4v4fER9-_HgefGLdvup2izeQFSFdNF60Mx8nkHlA-hNt_qy78YIW3uSyVuE41im79VjnJrpg/s640/linkage.PNG" width="640" /></a></div>
<div>
<br /></div>
<div>
Oops, now that's surprising. It's strange, that DefaultGC (ParOldGC) actually gets faster for objects containing *more* references to (random) other objects !! This is not a test error and reproducable.<br />
You probably now, that todays CPU's have a multi stage cache and that access to main memory is extremely expensive compared to cache hits, so locality (memory wise) is pretty important. Its obvious this also hit's Full GC when iterating the live set of objects.<br />
(see <a href="http://www.akkadia.org/drepper/cpumemory.pdf">http://www.akkadia.org/drepper/cpumemory.pdf</a> for more information you'd probably want to read about caches ;-) ).<br />
<br />
In contradiction to the CMS collector, ParOldGC ('DefaultGC') is able to compact the heap during Full GC. It seems it does this in a way optimizing the randomly linked object graph by moving objects close to objects referencing them (that's my speculative explanation, others are welcome ..). <br />
<br />
This observation leads to the following test:<br />
A comparion of FullGC of 6 million GCWrapper object graphs. One setup as linked list (linked in allocation order) and one with randomly choosen referenced objects.<br />
<br />
<blockquote class="tr_bq">
GCWrapper res = new GCWrapper(null);<br />
for ( int i = 0; i < len; i++ ) {<br />
res = new GCWrapper(res);<br />
}<br />
return res;</blockquote>
<div style="text-align: center;">
<b><span style="font-size: large;">vs</span></b></div>
<div>
<blockquote class="tr_bq">
GCWrapper res[] = new GCWrapper[len];<br />
for ( int i = 0; i < len; i++ ) {<br />
res[i] = new GCWrapper(null);<br />
}<br />
for ( int i = 0; i < len; i++ ) {<br />
res[i].referenced = res[((int) (Math.random() * len))];<br />
}<br />
return res;</blockquote>
</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYxnx8CQgz38WUKSQso2Uj9A_Nn4Ed0DLJgkh3kEC0i0m1E45Br0xcXoZ3Khovx_32mxxP4QGq7_WL_QxQoybe8HXWXhPxpu_QlGZETx8P3MJZUsCgsnH_sQBQu3h-At4G1Ocd8Xav86E/s1600/locality.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYxnx8CQgz38WUKSQso2Uj9A_Nn4Ed0DLJgkh3kEC0i0m1E45Br0xcXoZ3Khovx_32mxxP4QGq7_WL_QxQoybe8HXWXhPxpu_QlGZETx8P3MJZUsCgsnH_sQBQu3h-At4G1Ocd8Xav86E/s640/locality.PNG" width="640" /></a></div>
<div>
Boom ! The impact of locality is massive. This will also have a strong impact on performance of a java application accessing those data structures.<br />
<br />
<h2>
<b>Conclusion:</b></h2>
</div>
<div>
<blockquote class="tr_bq">
<i><b>Full GC duration depends on the number of objects allocated and the locality of their references. It does not depend that much on actual heap size.</b></i></blockquote>
</div>
<div>
<b><br /></b>
<b><br /></b></div>
<h2>
<b>Questionable design patterns regarding (OldGen) GC and overall locality</b></h2>
<div>
<br />
I think one can safely assume, that objects allocated at the same time have a high chance to be "near" each other (Note: CMS heap fragmentation may behave different if the application fragments the heap). So whenever statically allocated data is updated by refering to a newly allocated Object, chance is, that the new Object is far away from the referencee, which will cause cache misses. Especially the massive use of Immutables as favoured by many designs and (newer VM-based languages) actually is deadly regarding locality and OldGen GC.</div>
<div>
<ul>
<li>Lazy initialization/instantiation</li>
<li>Value classes. E.g. GC-wise its better to use a primitive long instead of Date.</li>
<li>Immutables (except preallocated Flyweight's)</li>
<li>Fine grained class design </li>
<li>Preferring Composition over Subclassing (many wrapper, delegate Objects)</li>
<li>JDK's HashMap uses entry objects. Use open addressed Hashmaps instead</li>
<li>Use of Long,Integer,Short instead of primitives in static data (unfortunately Java's Generics encourage this). Even if the VM does some optimization here (e.g. internal caching of Integer's), one can verify easily that these objects still have severe impact on GC duration</li>
</ul>
</div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com31tag:blogger.com,1999:blog-4356402223581768063.post-29792082005247839202013-07-07T11:54:00.000+02:002013-07-09T00:15:56.081+02:00Tuning and benchmarking Java 7's Garbage Collectors: Default, CMS and G1<br />
[i am not a native english speaker, so enable quirks mode pls ...]<br />
<br />
Due to trouble in a project with long GC pauses, I just had myself a deeper look into GC details. There is not that much acessible information/benchmarks on the Web, so I thought I might share my tests and enlightments ^^. Last time I tested GC some years ago I just came to the conclusion, that allocation of any form is evil in Java and avoided it as far as possible in my code.<br />
<br />
I will not describe the exact algorithms here as there is plenty of material regarding this. Understanding the algorithms in detail does not necessary help on predicting actual behaviour in real systems, so I will do some tests and benchmarks.<br />
<br />
The concepts of Garbage Collection in Hotspot are explained e.g. <a href="http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html" target="_blank">here</a>.<br />
In depth coverage of algorithms and parameters can be found <a href="http://blog.ragozin.info/2011/06/understanding-gc-pauses-in-jvm-hotspots.html" target="_blank">here</a>. I will cover GC from an empirical point of view here.<br />
<br />
The basic idea of multi generational Garbage Collection is, to collect newer ("younger") Objects more frequent using a "minor" (short duration/pause) Collection algorithm. Objects which survived one or more minor collections then move to the "OldSpace". The OldSpace is garbage collected by the "major" Garbage Collector. I will name them NewSpace and NewGC, OldSpace and OldGC.<br />
<br />
The NewGC Algorithm is pretty much the same amongst the 3 Garbage Collectors HotSpot provides. The Old Generation Collector ("OldGC") makes the difference.<br />
<br />
<ul>
<li>In the <b>Default Collector</b>, OldSpace is cleaned up using a "Stop the World" Mark-Sweep.</li>
<li><b>The Concurrent Mark&Sweep</b> Collector (<b>CMS</b>) uses (surprise!) a concurrent implementation of the Mark & Sweep Collection. This means the running application is not stopped during major GC. However it falls back to Stop-The-World GC if it can't keep up with applications allocation (promotion, tenuring) rate.</li>
<li><b>The G1</b> is also concurrent. It segments memory into equal chunks, trying to split the Garbage Collection Process into smaller pieces by collecting segments which are likely to contain a lot of garbage first. A more detailed description can be found <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/vm/G1.html" target="_blank">here</a>. It also falls back to Full-Stop-GC in case it can't keep up with application allocation rate. However the duration of those Full-GC pauses are of shorter duration compared to the other OldSpace collectors.</li>
</ul>
<br />
Note that the term "parallel GC" refers to collection algorithms which run multithreaded, not necessary in parallel to your application.<br />
<br />
<br />
<h3>
The Benchmark</h3>
<div>
<br /></div>
<div>
I wrote a small program which emulates most of the stuff Garbage Collectors have problems with:</div>
<div>
<br /></div>
<div>
<ul>
<li>4GB of statically allocated Objects which are rarely freed.<br />Problem for GC: with each major GC objects must be traversed, so the larger your reference data, cache's etc., the longer major GC will need for a full traversal collection.</li>
<li>A lot of temporary Object allocation of various size and age. "Age" refers to the amount of time the Object is referenced by the application.</li>
<li>Intentional partial replacements of pseudo-static data by new objects. This way I enforce "promotion" of objects to OldSpace, as they are long lived.</li>
</ul>
This is achieved by putting a lot of objects into a HashMap, then replace a fraction of it. Additionally the latency of a ~0,5 ms operation is measured and memorized to simulate processing of e.g. Requests. This way I get a distribution of VM/GC-related pauses. The benchmark runs for 5 minutes, so long term effects like heap fragmentation are <b>not </b>covered by the tests.</div>
<div>
<br /></div>
<div>
Note that this benchmark has an insane allocation rate and object age distribution. So results and VM tuning evaluated in this post illustrate the effects of some GC settings, its not cut & paste stuff, most of the sizings used to get this allocation greedy benchmark to work are way to big for real world applications.</div>
<div>
<br />
<br /></div>
<div>
<h3>
Real men use default settings ..</h3>
</div>
<div>
<br /></div>
<div>
Running the benchmark with -Xms8g -Xmx8g (the bench consumes 4g static, ~1g dynamic data) yields to quite disgusting results:</div>
<div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxzbokUaI9UtH_NG9TcH_4PxJYSdk-B1vJewRl_XHQKgifQ8fjJ2I7ovAbrgZO6ht5S0_FjpKzeUI8cOLugFz4gfXcYJA8FTZofdn6y4W6neS4wXyBqRIBOF8fY-LPSJx73uas3dtgXyY/s1600/8gdefaultnotweaks.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="152" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxzbokUaI9UtH_NG9TcH_4PxJYSdk-B1vJewRl_XHQKgifQ8fjJ2I7ovAbrgZO6ht5S0_FjpKzeUI8cOLugFz4gfXcYJA8FTZofdn6y4W6neS4wXyBqRIBOF8fY-LPSJx73uas3dtgXyY/s1600/8gdefaultnotweaks.png" width="640" /></a></div>
<br />
Each of the spikes represents a full GC with a duration of ~15 seconds ! During the full GC, the program is stopped. One can see, that the Garbage Collecor has some automatic calibration. Initially it did a pretty good job, but after first GC things get really bad. <br />
I've noticed this in several tests. Most of the time the automatic calibration improves things, but sometimes it's vice versa (see Enlightment #2).<br />
<br />
Hiccups ( [interval in milliseconds] => number of occurences ).<br />
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Courier New, Courier, monospace;">Iterations 9324, </span><span style="font-family: 'Courier New', Courier, monospace;">-Xmx8g -Xms8g </span><span style="font-family: Courier New, Courier, monospace;"><br />[0]<span class="Apple-tab-span" style="white-space: pre;"> </span>4930<br />[1]<span class="Apple-tab-span" style="white-space: pre;"> </span>4366<br />[2]<span class="Apple-tab-span" style="white-space: pre;"> </span>8<br />[7]<span class="Apple-tab-span" style="white-space: pre;"> </span>1<br />[14000]<span class="Apple-tab-span" style="white-space: pre;"> </span>1<br />[15000]<span class="Apple-tab-span" style="white-space: pre;"> </span>10<br />[16000]<span class="Apple-tab-span" style="white-space: pre;"> </span>4<br />[17000]<span class="Apple-tab-span" style="white-space: pre;"> </span>3<br />[18000]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span></span></blockquote>
Read like e.g '4930' requests done in 0 ms, 10 requests had a response time of 15 seconds. Throughput was 9324 requests.<br />
<br />
CMS did somewhat better, however there were also severe program-stopping GC's. Note the >10 times higher throughput.<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Iterations 115726, ‐XX:+UseConcMarkSweepGC -Xmx8g -Xms8g<br />[0]<span class="Apple-tab-span" style="white-space: pre;"> </span>52348<br />[1]<span class="Apple-tab-span" style="white-space: pre;"> </span>62317<br />[2]<span class="Apple-tab-span" style="white-space: pre;"> </span>157<br />[46]<span class="Apple-tab-span" style="white-space: pre;"> </span>7<br />[61]<span class="Apple-tab-span" style="white-space: pre;"> </span>13<br />[62]<span class="Apple-tab-span" style="white-space: pre;"> </span>127<br />[63]<span class="Apple-tab-span" style="white-space: pre;"> </span>234<br />[64]<span class="Apple-tab-span" style="white-space: pre;"> </span>164<br />[65]<span class="Apple-tab-span" style="white-space: pre;"> </span>81<br />[66]<span class="Apple-tab-span" style="white-space: pre;"> </span>55<br />[67]<span class="Apple-tab-span" style="white-space: pre;"> </span>45<br />[68]<span class="Apple-tab-span" style="white-space: pre;"> </span>42<br />[69]<span class="Apple-tab-span" style="white-space: pre;"> </span>25<br />[70]<span class="Apple-tab-span" style="white-space: pre;"> </span>28<br />[71]<span class="Apple-tab-span" style="white-space: pre;"> </span>16<br />[72]<span class="Apple-tab-span" style="white-space: pre;"> </span>11<br />[73]<span class="Apple-tab-span" style="white-space: pre;"> </span>5<br />[74]<span class="Apple-tab-span" style="white-space: pre;"> </span>12<br />[14000]<span class="Apple-tab-span" style="white-space: pre;"> </span>7<br />[15000]<span class="Apple-tab-span" style="white-space: pre;"> </span>4<br />[17000]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span></blockquote>
<br />
<span style="font-family: inherit;">G1 does the best job here (still 10 inacceptable full stop GC's..)</span><br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Iterations 110052</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[0]<span class="Apple-tab-span" style="white-space: pre;"> </span>37681<br />[1]<span class="Apple-tab-span" style="white-space: pre;"> </span>67201<br />[2]<span class="Apple-tab-span" style="white-space: pre;"> </span>1981<br />[3]<span class="Apple-tab-span" style="white-space: pre;"> </span>725<br />[4]<span class="Apple-tab-span" style="white-space: pre;"> </span>465<br />[5]<span class="Apple-tab-span" style="white-space: pre;"> </span>306<br />[6]<span class="Apple-tab-span" style="white-space: pre;"> </span>217<br />[7]<span class="Apple-tab-span" style="white-space: pre;"> </span>182<br />[8]<span class="Apple-tab-span" style="white-space: pre;"> </span>113<br />[9]<span class="Apple-tab-span" style="white-space: pre;"> </span>105<br />[39]<span class="Apple-tab-span" style="white-space: pre;"> </span>13 </span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">(skipped some intervals of minor occurence count)</span><span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br />[100]<span class="Apple-tab-span" style="white-space: pre;"> </span>288<br />[200]<span class="Apple-tab-span" style="white-space: pre;"> </span>23<br />[300]<span class="Apple-tab-span" style="white-space: pre;"> </span>6<br />[1000]<span class="Apple-tab-span" style="white-space: pre;"> </span>11<br />[12000]<span class="Apple-tab-span" style="white-space: pre;"> </span>5<br />[13000]<span class="Apple-tab-span" style="white-space: pre;"> </span>5</span></blockquote>
<span style="font-family: inherit;">Hm .. pretty bad. Now, let's have a look into the GC internals:</span></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNodLQE9aAMcEsd5BUVZic8GQlmzHJ5EYAlhau830dv2urIMwH0Dmo-GakNOh396zcnFtbxsX0d3fnuF3wntyQO0pVo22auIJh4oN15KiuMiV2s_APJlOZ10lW96855Z38guJbFRXQ4hY/s1600/gc_intern.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNodLQE9aAMcEsd5BUVZic8GQlmzHJ5EYAlhau830dv2urIMwH0Dmo-GakNOh396zcnFtbxsX0d3fnuF3wntyQO0pVo22auIJh4oN15KiuMiV2s_APJlOZ10lW96855Z38guJbFRXQ4hY/s1600/gc_intern.PNG" width="640" /></a></div>
<div>
<br /></div>
<div>
If OldSpace (big green one) is full, the application gets a full-stop GC. In order to avoid Full GC, we need to reduce the promotion rate to OldSpace. An object gets promoted if it is alive (aka referenced) for a longer time than NewSpace (=Eden+Survivor Spaces) holds it.</div>
<div>
<br /></div>
<div>
So time for </div>
<div>
<br /></div>
<div>
<blockquote class="tr_bq">
<i><b>Finding #1:</b> </i></blockquote>
<blockquote class="tr_bq">
<b><i>It is key to reduce the promotion rate from young gen to OldSpace. The promotion rate to OldSpace needs to be lowered in order to avoid Full-GC's. In case of concurrent OldSpace GC (CMS,G1), this will enable collectors to keep up with allocation rate.</i></b></blockquote>
<br /></div>
<div>
</div>
<h3>
</h3>
<h3>
Tuning the Young Generation</h3>
<div>
<br />
All 3 Collectors avaiable in Java 7 will profit of a proper NewGC setup.<br />
<br /></div>
<div>
Promotion happens if:</div>
<div>
<ul>
<li>The "survivor space" (S0, S1) is full. <br />The size of Survivor vs Eden is specified with the -XX:SurvivorRatio=N. A large Eden will increase throughput (usually) and will catch ultra-short lived temporary Objects better. However large Eden means small Survivor Spaces, so middle aged Objects might get promoted too quickly to OldSpace then, putting load on OldSpace GC.</li>
<li><span style="background-color: white; white-space: pre-wrap;"><span style="font-family: inherit;">Survivors have survived more than</span></span><span style="background-color: white; font-family: arial, sans, sans-serif; font-size: 12.727272033691406px; white-space: pre-wrap;"> </span><span style="color: #333333; font-family: 'Courier New'; font-size: 10pt;">-XX:Max</span><span style="color: #333333; font-family: 'Courier New'; font-size: 10pt;">TenuringThreshold=N (Default 15) </span><span style="color: #333333;"><span style="font-family: inherit;"><span style="font-family: inherit;">minor collections.</span> Unfortunately I did not find an option to give a lower bound for this value, so one can specify a maximum here only. </span></span>-XX:InitialTenuringThreshold might help, however I found the VM will choose lower values anyway in case.</li>
</ul>
<span style="color: #333333;"></span><br />
<div>
<span style="color: #333333;"><span style="color: #333333;"><br /></span></span></div>
<span style="color: #333333;">
The following actions will reduce promotion (by encouraging survivors to live longer in young generation)</span></div>
<ul><span style="font-family: inherit;">
<li><span style="font-family: inherit;">Decreasing -XX:SurvivorRatio=N to lower values than 8 (this actually increases the size of survivor spaces and decreases Eden size). <br />Effect is that survivors will stay for a longer time in young gen (if there is sufficient size)<br />This will reduce throughput as survivors are copied with each minor GC between S0,S1.</span></li>
<li><span style="font-family: inherit;">Increase the overall size of young generation with -XX:NewRatio=N. <br />"1" means, young generation will use 50% of your heap, 2 means it will use 33% etc. A larger young gen reduces heap size for long-lived objects but will reduce the number of minor GCs and increase the size avaiable for survivors.</span></li>
</span>
<li><span style="font-family: inherit;"><span style="font-family: inherit;">Increase -XX:MaxTenuringThreshold=N to values > 15.<br />Of course this only reduces promotion, if the survivor space is large enough. Additonally this is only an upper bound, so the VM might choose a lower value regardless of your setting (you can also try </span></span>-XX:InitialTenuringThreshold<span style="font-family: inherit;"><span style="font-family: inherit;">). </span></span></li>
<span style="font-family: inherit;">
<li><span style="font-family: inherit;">Increasing overall VM heap will help (in fact more GB always help :-) ), as this will increase young generation (Eden+Survivor) and OldSpace size. An increase in OldSpace size reduces the number of required major GC's (or give concurrent OldSpace collectors more headroom to complete a major GC concurrent, pauseless).</span></li>
</span></ul>
<span style="font-family: inherit;">
</span>
<br />
<div>
It depends on the allocation behaviour of your application which of this actions will have effect.<br />
<br />
<br />
<blockquote class="tr_bq">
<i><b>Finding #2:</b> </i></blockquote>
<blockquote class="tr_bq">
<b><i>When adjusting Survivor Ratio and/or MaxTenuringThreshold manually, always switch off auto adjustment with<br /> <span style="font-family: Courier New, Courier, monospace;">-XX:-UseAdaptiveSizePolicy</span></i></b></blockquote>
<br />
<br />
Below the effects of NewSpace settings on memory sizing. Note that the sizing of NewSpace directly reduces avaiable space in OldSpace So if your application holds a lot of statically allocated data, increasing size of young generation might require you to increase overall memory size.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkSpnlm8uu9JVz_Vcs6mtv6a-vFCrGIvF48ka78oPDenfhPplR6CTvd5d520xZDQh13526O1fwH-3WmXFDqjmHnSCJH4OFBoUT-DOfWxZUu73HyLGn9Jm8tfoYiDGj_HoPbh9xMIaO09o/s1600/gc-sizing.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkSpnlm8uu9JVz_Vcs6mtv6a-vFCrGIvF48ka78oPDenfhPplR6CTvd5d520xZDQh13526O1fwH-3WmXFDqjmHnSCJH4OFBoUT-DOfWxZUu73HyLGn9Jm8tfoYiDGj_HoPbh9xMIaO09o/s1600/gc-sizing.PNG" width="640" /></a></div>
</div>
<div>
<br /></div>
<div>
Note: In practice one would evaluate the required size of NewSpace and specify it with <span style="color: #333333; font-family: 'Courier New'; font-size: 10pt;">-XX:NewSize=X </span><span style="color: #333333; font-family: 'Courier New'; font-size: 10pt;">-XX:MaxNewSize=X </span>absolutely<span style="color: #333333; font-size: 10pt;"><span style="font-family: inherit;">. </span></span>This way changing -Xmx will affect OldSpace size only and will not mess up absolute Survivor Space sizes by applying ratios.<br />
<br />
Actually the Default GC is the most useful collector to use in order to profile NewSpace setup, since there is no 2cnd background collector bluring OldSpace growth.</div>
<div>
<br />
<b>
Survivor Sizing</b></div>
<div>
<br />
The most important thing is, to figure out a good sizing (absolute, not ratio) for the survivor spaces and the promotion counter (MaxTenuringThreshold). Unfortunately there is a strong interaction between Eden size and MaxTenuringThreshold: If Eden is small, then Objects are put into survivor spaces faster, additional the tenuring counter is incremented more quickly. This means if Eden is doubled in size, you probably want to decrease your MaxTenuringThreshold and vice versa. This gets even more complicated as the number of Eden GC (=minor GC) also depends on application allocation rate.<br />
<br /></div>
<div>
The optimal survivor size is large enough to hold middle lived Objects under full application load without promoting them due to size shortage.<br />
<br />
<ul>
<li>If survivor space is too large, its just a waste of memory which could be given to Eden instead. Additionally there is a correlation between survivor size and minor GC pauses (throughput degradation, jitter) [Fixme: to be proven].</li>
<li>If survivor space is too small, Objects will be promoted to OldSpace too early even if TenuringThreshold is not reached yet.</li>
</ul>
<br />
<b>
MaxTenuringThreshold</b><br />
<br /></div>
<div>
This defines how many minor GC's an Object may survive in SurvivorSpace before getting tenured to OldSpace. Again you have to optimize this under max application load, as without load there are fewer minor GC's so the "survived"-counters will be lower. Another issue to think of is that Eden size also affects the frequency of minor GC's. The VM will handle your value as an upper bound only and will automatically use lower values if it thinks these are more appropriate.<br />
<br />
<ul>
<li>If MaxTenuringThreshold is too high, throughput might suffer, as non-temporary Objects will be subject to minor collection which slows down application. As said, the VM automatically corrects that.</li>
<li>If MaxTenuringThreshold is too low, temporary Objects might get promoted to OldSpace too early. This can hamper the OldSpace collector and increase the number of OldSpace GC's. If the promotion rate gets too high, even concurrent Collecors (CMS, G1) will do a full-stop-GC.</li>
</ul>
<br />
If in doubt, set MaxTenuringThreshold too high, this won't have a significant impact on application performance in practice. <br />
It also strongly depends on the coding quality of the application: if you are the kind of guy preferring zero allocation programming, even a MaxTenuringThreshold=0 might be adequate (there is also kind of "alwaysTenure" option). The other extreme is "return new HashBuilder().computeHash(this);"-style (some alternative VM language produce lots of short to mid-lived temporary Garbage) where a settings like '30' or higher (which most often means: keep survivors as long there is room in SurvivorSpace) might be required.<br />
<br />
<br />
Common sign of too slow promotion (MaxTenuringThreshold too high, Survivor size too high):<br />
(<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">-Xmx12g -Xms12g -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=3 -XX:NewRatio=2 -XX:MaxTenuringThreshold=20</span>)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwpfG18nT79zQ8jy5BLJZuUPzWdTIHnjdKpgf0wz0Wi8X-k2XiWnjs7_SyAEfdjFVkYe4RzuyblvVrK81OoN_yrgbmAaNHrNmAypIsWrJYqoqwR3HrAk9g_J2H56udw-MV-rlGHcC9kBY/s1600/survivortoolarge.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="136" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwpfG18nT79zQ8jy5BLJZuUPzWdTIHnjdKpgf0wz0Wi8X-k2XiWnjs7_SyAEfdjFVkYe4RzuyblvVrK81OoN_yrgbmAaNHrNmAypIsWrJYqoqwR3HrAk9g_J2H56udw-MV-rlGHcC9kBY/s1600/survivortoolarge.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: x-small;"><i>bad survivor, TenuringThreshold setting</i></span></div>
<div>
<br /></div>
<div>
Initially it looks like there are no Objects promoted to OldSpace, as they actually "sit" in Survivor Space. Once the survivor spaces get filled, survivors are tenured to old Space resulting in a sudden increase of promotion rate. (5 minute chart of benchmark running with default GC, actually the application does not allocate more memory, it just constantly replaces small fractions of initially allocated pseudo-static Objects). This will probably confuse concurrent OldSpace Collectors, as they will start concurrent collection too late. Beware: Clever project manager's might bug you to look for memory leaks in your application or to plow through the logs to find out "what happened 14:36 when memory consumption all over a sudden starts to rise".</div>
<div>
<br /></div>
<div>
Same benchmark with better settings (<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">-Xmx12g -Xms12g -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=4 -XX:NewSize=4g -XX:MaxNewSize=4g -XX:MaxTenuringThreshold=15</span>):</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHiamArLFUfMkw7w3R4WpRjYtcPUi4woIy-4M28g7SBh7PJUgWYYA0xVxLRgDZBgImIhE3LWLuuFjGeBKYH8FExUwwl56o35Y6Qz96bRpgob8VBQR13bmwGNYvuDiZ4m5ltt6bhMYwaKU/s1600/survivor_right.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="165" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHiamArLFUfMkw7w3R4WpRjYtcPUi4woIy-4M28g7SBh7PJUgWYYA0xVxLRgDZBgImIhE3LWLuuFjGeBKYH8FExUwwl56o35Y6Qz96bRpgob8VBQR13bmwGNYvuDiZ4m5ltt6bhMYwaKU/s1600/survivor_right.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i style="font-size: small;">better survivor, TenuringThreshold setting</i></div>
<div>
<br /></div>
<div>
The promotion rate now reflects the actual allocation rate of long-lived objects. Since there is no concurrent OldSpace Collector in the default GC, it looks like a permanent growth. Once the limit is reached, a Full-GC will be triggered and will clean up unused long lived Objects. A concurrent collector like CMS, G1 will now be able to detect promotion rate and keep up with it.</div>
<br />
<b>
Eden Size</b><br />
<b><br /></b>
Eden Size directly correlates with throughput as most java applications will create a lot of temporary objects.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirrRzmnTkmpqVzEghGhYfV5AkmY06VZFINjBm58nU3W9gvGTblpD2aQGhgi5m1FvbvskuvduS0l9wImgzHerG744Fw-zjnbZQN4EUcg7HECUtv3-uSZJ8vcltrLveJrqb53DpLJYEDugs/s1600/throughputeden.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="305" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirrRzmnTkmpqVzEghGhYfV5AkmY06VZFINjBm58nU3W9gvGTblpD2aQGhgi5m1FvbvskuvduS0l9wImgzHerG744Fw-zjnbZQN4EUcg7HECUtv3-uSZJ8vcltrLveJrqb53DpLJYEDugs/s1600/throughputeden.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i><span style="font-size: x-small;">as measured with high allocation rate application</span></i></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdtfZpW2H8-lP2MCTtIvMAzd2H1OyiFqJbZ38aN60oR2Ae2IYzbeI_O8Z9BbKVnl5-1KRistDHJWdAJXWfNiwDucVs8vQkxpYfgW4QqkkSPNLRIyfXBxuYVixIWaSGFvwNYZMP6aZT-Wo/s1600/pauseseden.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdtfZpW2H8-lP2MCTtIvMAzd2H1OyiFqJbZ38aN60oR2Ae2IYzbeI_O8Z9BbKVnl5-1KRistDHJWdAJXWfNiwDucVs8vQkxpYfgW4QqkkSPNLRIyfXBxuYVixIWaSGFvwNYZMP6aZT-Wo/s1600/pauseseden.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: x-small;"><i>Eden size: red: 642MB, blue: 1,88GB, yellow: 3,13GB, green: 4,39GB</i></span></div>
<br />
As one can see, Eden size correlates with number, not the length of minor collections.<br />
<br />
Settings used for benchmark (Survivor spaces are kept constant, only Eden is enlarged):<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=1 -XX:NewSize=1926m -XX:MaxNewSize=1926m -XX:MaxTenuringThreshold=40 <br />= 642m Eden<br /><br />-XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=3 -XX:NewSize=3210m -XX:MaxNewSize=3210m -XX:MaxTenuringThreshold=15 <br />= 1,88g Eden<br /><br />-XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=5 -XX:NewSize=4494m -XX:MaxNewSize=4494m -XX:MaxTenuringThreshold=12 <br />= 3,13g Eden<br /><br />-XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=7 -XX:NewSize=5778m -XX:MaxNewSize=5778m -XX:MaxTenuringThreshold=8 <br />= 4,39g Eden (heap size 14Gb required)</span><br />
<br />
<blockquote class="tr_bq">
<i><b>Finding #3: </b></i><br />
<ul>
<li><i><b>Eden size strongly correlates with throughput/performance for common java applications (with common allocation patterns). The difference can be massive.</b></i></li>
</ul>
<ul>
<li><i><b>Eden size, Allocation rate, Survivor size and TenuringThreshold are interdependent. If one of those factors is modified, the others need readjustment</b></i></li>
</ul>
<ul>
<li><i><b>Ratio-based options can be dangerous and lead to false assumptions. NewSpace size should be specified using absolute settings (-XX:NewSize=)</b></i></li>
</ul>
<ul>
<li><i><b>Wrong sizing can lead to strange allocation and memory consumption patterns</b></i> </li>
</ul>
</blockquote>
<div>
<br /></div>
<h3>
Tuning OldSpace Garbage Collectors</h3>
<div>
<br /></div>
<b>Default GC</b><br />
<b><br /></b>
<br />
<div>
Default GC has a Full-Stop-GC (>15 seconds with the benchmark), so there is not much to do. If you want to run your application with Default GC (e.g. because of high throughput requirements), your only choice is to tune NewSpace GC very agressively, then throw tons of Heap to your application in order to avoid Full-GC during the day. If your system is 24x7 consider triggering a full GC using System.gc() at night if you expect the system load to be low.</div>
<div>
Another possibility would be to even size the VM bigger than your physical RAM, so tenured Objects are written to swap disk. However you have to be sure then no Full-GC is triggered ever, because duration of Full-GC will go into the minutes then. I have not tried this.</div>
<div>
Ofc one can improve things always by coding less memory intensive, however this is not the topic of this post.</div>
<br /></div>
<div>
<b>
Concurrent Mark & Sweep (CMS)</b><br />
<b><br /></b>
<br />
<div>
The CMS Collector does a pretty good job as long your promotion rate is not too high (which should not be the case if you optimized that as described above).</div>
<div>
One Key setting of CMS Collector is, when to trigger a concurrent full GC. If it is triggered too late, it might not be able to finish in time and a Full-Stop-GC will happen. If you know your application has like 30% statically allocated data you might want to set this to 30% like -XX:+UseCMSInitiatingOccupancyOnly<br />
-XX:CMSInitiatingOccupancyFraction=30. In practice I always start with a value of 0, then experiment with higher values once everything (NewSpace, OldSpace) is calibrated to operate without triggering FullGC under load.</div>
<br />
When i copy the settings evaluated in the "NewGen Tuning" settings straight forward, the result will be a permanent Full-GC. Why ? <br />
Because CMS requires more heap than the default GC. It seems like the same data structures just require ~20-30% more memory. So we just have to multiply NewGC settings evaluated from Default GC with 1.3.<br />
Additionally, a good start is to let the concurrent Mark & Sweep run all the time.<br />
<br />
So I go with (copied 2cnd NewSpace config from above and multiplied):<br />
<br />
‐XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly <br />
-XX:CMSInitiatingOccupancyFraction=10 -Xmx12g -Xms12g -XX:-UseAdaptiveSizePolicy <br />
-XX:SurvivorRatio=3 -XX:NewSize=4173m -XX:MaxNewSize=4173m <br />
-XX:MaxTenuringThreshold=15<br />
<br />
The CMS is barely able to keep up with promotion rate, throughput is 300.000 which is acceptable given that cost of (in contradiction to tests above) Full-GC is included.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsN80LBcxejVSIS2WL0NfPlrkvsNa3OpIl0EzMFFpq3Nql09XdVCXGxobPsWfrHKwfineQOlSPs6qzlj9-UJotEbemw8UxMaldKBki-9dI9FS6B1MCLzIQG3jUE7-qr6wwcINd8g1-o0Y/s1600/cmsoldgen.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="264" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsN80LBcxejVSIS2WL0NfPlrkvsNa3OpIl0EzMFFpq3Nql09XdVCXGxobPsWfrHKwfineQOlSPs6qzlj9-UJotEbemw8UxMaldKBki-9dI9FS6B1MCLzIQG3jUE7-qr6wwcINd8g1-o0Y/s1600/cmsoldgen.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i><span style="font-size: x-small;">CMS </span></i><span style="text-align: left;"><i><span style="font-size: x-small;">OldSpace</span></i></span><i><span style="font-size: x-small;"> collection can keep up with promotion rate </span></i><i><span style="font-size: x-small;">barely</span></i></div>
<br />
We can see from Visual GC (an excellent jVisualVM plugin), that OldSpace size is on the edge of triggering a full GC. In order to improve throughput, we would like to increase eden. Unfortunately there is not enough headroom in OldSpace, as a reduction of OldSpace would trigger Full-Stop-GC's.<br />
Reducing the size of Surviver Spaces is not an option, as this would result in a higher tenured Object rate and again trigger Full GC's. The only solution is: More Memory.<br />
Comparing throughput with the Default GC test above is not fair, as the Default GC would run into Full-Stop-GC's for sure, if the test would run for a longer time than 5 minutes.<br />
On a side note: the memory chart of jVisual VM's (and printouts by -verbose:gc) does not tell you the full story as you cannot see the fraction of used memory in OldSpace.<br />
<br />
Ok, so lets add 2 more Gb to be able to increase eden resulting in<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly <br />-XX:CMSInitiatingOccupancyFraction=10 -Xmx14g -Xms14g -XX:-UseAdaptiveSizePolicy <br />-XX:SurvivorRatio=4 -XX:NewSize=5004m -XX:MaxNewSize=5004m <br />-XX:MaxTenuringThreshold=15</span><br />
<br />
(Eden size is 3,25 GB).<br />
This increases throughput by 10% to 330.000.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRin5NIinL6XuH1RybThV-NS0PTh8AzpHyKD51R3bODMaWt0X_QrWVqBOjjeMKsOnIZ0j95Jh7Y_W5meGzwIAXrXeWexSObTqU0W2r-qhXQmBY1QDyb_RC0mxrWs9ndAE_3Em0v07Ef28/s1600/cmspauses.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRin5NIinL6XuH1RybThV-NS0PTh8AzpHyKD51R3bODMaWt0X_QrWVqBOjjeMKsOnIZ0j95Jh7Y_W5meGzwIAXrXeWexSObTqU0W2r-qhXQmBY1QDyb_RC0mxrWs9ndAE_3Em0v07Ef28/s1600/cmspauses.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i><span style="font-size: x-small;">blue is Heap 12g, Eden 2,4g, red is Heap 14g, Eden 3,2g</span></i></div>
<br />
The longest pause noted in processing were 400ms. Exact Reponse time distribution:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">-XX:+UseConcMarkSweepGC <br />-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=10 <br />-Xmx14g -Xms14g <br />-XX:-UseAdaptiveSizePolicy <br />-XX:SurvivorRatio=4 -XX:NewSize=5004m -XX:MaxNewSize=5004m <br />-XX:MaxTenuringThreshold=15</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Iterations 327175</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[0]<span class="Apple-tab-span" style="white-space: pre;"> </span>146321</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[1]<span class="Apple-tab-span" style="white-space: pre;"> </span>179968</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[2]<span class="Apple-tab-span" style="white-space: pre;"> </span>432</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[3]<span class="Apple-tab-span" style="white-space: pre;"> </span>33</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[14]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[70]<span class="Apple-tab-span" style="white-space: pre;"> </span>2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[82]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[83]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[85]<span class="Apple-tab-span" style="white-space: pre;"> </span>2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[87]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[100]<span class="Apple-tab-span" style="white-space: pre;"> </span>3</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[200]<span class="Apple-tab-span" style="white-space: pre;"> </span>355</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[300]<span class="Apple-tab-span" style="white-space: pre;"> </span>44</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[400]<span class="Apple-tab-span" style="white-space: pre;"> </span>11</span><br />
<br />
<br />
<blockquote class="tr_bq">
<b><i>Finding #4:</i></b><br />
<ul>
<li><b><i>Eden size still correlates with throughput for CMS</i></b></li>
</ul>
<ul>
<li><b><i>Compared with the Default Hotspot GC all NewGen sizes must be multiplied by 1.3</i></b></li>
</ul>
<ul>
<li><b><i>CMS also requires 20%-30% more Heap for identical data structures</i></b></li>
</ul>
<ul>
<li><b><i>No Full-Stop-GCs at all if you can provide the necessary amount of memory (large NewSpace + HeadRoom in OldSpace)</i></b></li>
</ul>
</blockquote>
<br />
<b>
The G1 Collector</b><br />
<b><br /></b>
<br />
<div>
The G1 collector is different. Each segment of G1 has its own NewSpace, so absolute values set with <span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">-XX:NewSize=5004m -XX:MaxNewSize=5004m</span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;"> </span>are actually ignored (don't ask me how long i fiddled with G1 and the NewSize parameters until I got this). Additionally VisualGC does not work for G1 Collector.</div>
<div>
Anyway, we still know that the benchmark succeeds in being one of the most greedy java programs out there, so ..</div>
<div>
<ul>
<li>we like large survivor spaces and a large NewSpace to reduce promotion to OldSpace</li>
<li>we like large Eden to trade memory efficiency against throughput. Memory is cheap.</li>
</ul>
</div>
Unfortunately not much is known about the efficiency of the G1 OldSpace collector, so the only possibility is testing. If the G1 OldSpace collector is more efficient than CMS OldSpace collector, we could decrease survivor space in favour of bigger eden spaces, because with a potentially more efficient G1 OldSpace collector, we may afford a higher promotion rate.</div>
<div>
<br /></div>
<div>
Here we go:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-Xmx12g -Xms12g -XX:+UseG1GC -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=1 -XX:NewRatio=1 -XX:MaxTenuringThreshold=15 </span></div>
<div>
yields:</div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Iterations 249372</span><br />
(values < 100ms skipped)</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[100]<span class="Apple-tab-span" style="white-space: pre;"> </span>10</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[200]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[400]<span class="Apple-tab-span" style="white-space: pre;"> </span>4</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[500]<span class="Apple-tab-span" style="white-space: pre;"> </span>198</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[600]<span class="Apple-tab-span" style="white-space: pre;"> </span>15</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[700]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span></div>
<br />
well, that's pretty good as first effort. Lets try to increase decrease survivors and start concurrent G1 earlier ..</div>
<div>
<br /></div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-Xmx12g -Xms12g -XX:+UseG1GC -XX:-UseAdaptiveSizePolicy <b>-XX:SurvivorRatio=2</b> -XX:NewRatio=1 -XX:MaxTenuringThreshold=15 <span style="background-color: white; white-space: pre-wrap;">-XX:InitiatingHeapOccupancyPercent=0</span></span><br />
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Iterations 233262</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[100]<span class="Apple-tab-span" style="white-space: pre;"> </span>3</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[200]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[400]<span class="Apple-tab-span" style="white-space: pre;"> </span>81</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[500]<span class="Apple-tab-span" style="white-space: pre;"> </span>96</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[600]<span class="Apple-tab-span" style="white-space: pre;"> </span>3</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[700]<span class="Apple-tab-span" style="white-space: pre;"> </span>5</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[800]<span class="Apple-tab-span" style="white-space: pre;"> </span>2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[1000]<span class="Apple-tab-span" style="white-space: pre;"> </span>18</span><br />
<div>
<br /></div>
<br />
err .. nope. We can see that decreasing survivor space puts more load on G1 OldSpace GC as the number of 1 second area pauses increased significantly. Just another blind shot relaxing survivors and increase G1 OldSpace load ..<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-Xmx12g -Xms12g -XX:+UseG1GC -XX:-UseAdaptiveSizePolicy <b>-XX:SurvivorRatio=6</b> -XX:NewRatio=1 -XX:MaxTenuringThreshold=15 <span style="background-color: white; white-space: pre-wrap;">-XX:InitiatingHeapOccupancyPercent=0</span></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Iterations 253293</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[100]<span class="Apple-tab-span" style="white-space: pre;"> </span>5</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[200]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[400]<span class="Apple-tab-span" style="white-space: pre;"> </span>186</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[500]<span class="Apple-tab-span" style="white-space: pre;"> </span>2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[600]<span class="Apple-tab-span" style="white-space: pre;"> </span>9</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[700]<span class="Apple-tab-span" style="white-space: pre;"> </span>5</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[1000]<span class="Apple-tab-span" style="white-space: pre;"> </span>16</span><br />
<br />
same throughput as intial trial, but larger pauses .. so I still need to look for large survivor and eden space. The only way to influence absolute NewSpace size is changing segment size (I take max size ofc).<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-Xmx12g -Xms12g -XX:+UseG1GC -XX:-UseAdaptiveSizePolicy <b>-XX:SurvivorRatio=1</b> -XX:NewRatio=1 -XX:MaxTenuringThreshold=15 <b>-XX:G1HeapRegionSize=32m</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Iterations 275464</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[100]<span class="Apple-tab-span" style="white-space: pre;"> </span>37</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[300]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[400]<span class="Apple-tab-span" style="white-space: pre;"> </span>5</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[500]<span class="Apple-tab-span" style="white-space: pre;"> </span>189</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[600]<span class="Apple-tab-span" style="white-space: pre;"> </span>16</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[900]<span class="Apple-tab-span" style="white-space: pre;"> </span>1</span><br />
<br />
Bingo ! Unfortunately all known tuning options are exploited now .. except main Heap size ..<br />
<div>
<br /></div>
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>-Xmx14g -Xms14g</b> -XX:+UseG1GC -XX:SurvivorRatio=1 -XX:NewRatio=1 -XX:MaxTenuringThreshold=15 </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">-XX:-UseAdaptiveSizePolicy </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">-XX:G1HeapRegionSize=32m</span><br />
<div>
<b><br /></b></div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Iterations 296270<br />[100] 58<br />[200] 3<br />[300] 2<br />[400] 4<br />[500] 160<br />[600] 15<br />[700] 11</span><br />
<div>
<b><br /></b></div>
This is slightly worse in latency and througput than a proper configured CMS. One last shot in order to figure out behaviour regarding memory size constraints (at least real peak size of the bench is round 4.5g ... sigh):</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>-Xmx8g -Xms8g</b> -XX:+UseG1GC -XX:SurvivorRatio=1 <b>-XX:NewRatio=2</b> -XX:MaxTenuringThreshold=15 -XX:-UseAdaptiveSizePolicy -XX:G1HeapRegionSize=32m</span><br />
<div>
<br /></div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Iterations 167721<br />[100] 37<br />[200] 1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[300] 3<br />[400] 297<br />[500] 7<br />[600] 9<br />[700] 5<br />[1000] 15<br />[2000] 1 </span><br />
<br />
<br />
<div>
It seems G1 excels in memory efficiency without having to do too long pauses (CMS falls into permanent full GC if I try to run the bench with 8g heap). This can be important in memory constrained server environments and at client side. </div>
<div>
<br /></div>
<div>
<br />
<blockquote class="tr_bq">
<b><i>Finding #5:</i></b><br />
<ul>
<li><b><i>Absolute settings for NewSpace size are ignored by G1. Only the ratio based options are acknowledged.</i></b></li>
</ul>
<ul>
<li><b><i>In order to influence eden/survivor space, one can increase/decreas segment size up to 32m. This also gave best throughput (possibly because of larger eden)</i></b></li>
</ul>
<ul>
<li><b><i>G1 can handle memory constraints the best without extremely long pauses</i></b></li>
</ul>
<ul>
<li><b><i>Latency distribution is ok, but behind CMS</i></b></li>
</ul>
</blockquote>
<br /></div>
<h3>
Conclusion</h3>
<div>
<b>Disclaimer: </b><br />
<br />
<ul>
<li>The benchmark used is worst case ever regarding allocation rate and memory waste. So take any finding with a grain of salt, when applying GC optimizations to your program.</li>
<li>I did not drive long term tests. All tests ran for 5 minutes only. Due to the extreme allocation rate of the benchmark, 5 minute benchmark is likely aequivalent to an hour operation of a "real" program. Anyway in an application with lower allocation rate, concurrent collectors will have more time to complete GC's concurrent, so you probably never will need an eden size of 4Gb in practice :).<br />I will provide long term runs in a separate post (maybe :) ).</li>
</ul>
<br />
<br />
<br />
<b>Default GC (Serial Mark&Sweep, Serial NewGC)</b> shows highest throughput as long no Full GC is triggered. <br />
If your system has to run for a limited amount of time (like 12 hours) and you are willing to invest into a very careful programming style regarding allocation; keep large datasets Off-Heap, DefaultGC can be the best choice. Of course there are applications which are ok with some long GC pauses here and there.</div>
<div>
<br /></div>
<div>
<b>CMS </b>does best in pause-free low latency operation as long you are willing to throw memory at it. Throughput is pretty good. Unfortunately it does not compact the heap, so fragmentation can be an issue over time. This is not covered here as it would require to run real application tests with many different Object sizes for several days. CMS is still way behind commercial low-latency solutions such as Azul's Zing VM.</div>
<div>
<br /></div>
<div>
<b>G1 </b>excels in robustness, memory efficiency with acceptable throughput.<b> </b>While CMS and DefaultGC react to OldSpace overflow with Full-Stop-GC of several seconds up to minutes (depends on Heap size and Object graph complexity), G1 is more robust in handling those situations. Taking into account the benchmark represents a worst case scenario in allocation rate and programming style, the results are encouraging.</div>
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGKvPNwmiHtcUvIQ3voGAEIPEhwj_KCox5q4DTXVpCo3PHBdddJusQijs9AY_irjZbZaY9nOXWvEkPo0FQQuryZbUvwV4lqlBfpFKxLhQsihg8IEJia-2hZEimANZHKp4GKlQERLTQAsE/s1600/latencyall.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGKvPNwmiHtcUvIQ3voGAEIPEhwj_KCox5q4DTXVpCo3PHBdddJusQijs9AY_irjZbZaY9nOXWvEkPo0FQQuryZbUvwV4lqlBfpFKxLhQsihg8IEJia-2hZEimANZHKp4GKlQERLTQAsE/s1600/latencyall.PNG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: x-small;"><i>Blue:CMS, Red: Default GC, Yellow: G1</i></span></div>
<div>
<br /></div>
Note: G1 has the lowest number of pauses. (Default GC has been tweaked to not do >15 second Full GC during test duration)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM-1PXZhyphenhyphen27hztHTZolErwFY6hXeyhMF8-8W3xaaxotj5V5ZUBUKB44XMXn8JmjDa0AnmHrayJJdw3B39k-5C1DPn5JkhrYgQHMW5MwRVbFdKCf4swRcqsgIux8tsNA7R_KjZ61STfB44/s1600/through.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="192" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM-1PXZhyphenhyphen27hztHTZolErwFY6hXeyhMF8-8W3xaaxotj5V5ZUBUKB44XMXn8JmjDa0AnmHrayJJdw3B39k-5C1DPn5JkhrYgQHMW5MwRVbFdKCf4swRcqsgIux8tsNA7R_KjZ61STfB44/s1600/through.PNG" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i><span style="font-size: x-small;">Default GC has been tweaked to not Full GC during test duration</span></i></div>
<div>
<b><br /></b></div>
<div>
<b><br /></b></div>
<div>
<b><br /></b></div>
<h3>
<b>The Benchmark source</b></h3>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">public class FSTGCMark {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> static class UseLessWrapper {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> Object wrapped;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> UseLessWrapper(Object wrapped) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> this.wrapped = wrapped;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> static HashMap map = new HashMap();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> static int hmFillRange = 1000000 * 30; //</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> static int mutatingRange = 2000000; //</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> static int operationStep = 1000;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int operCount;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int milliDelayCount[] = new int[100];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int hundredMilliDelayCount[] = new int[100];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int secondDelayCount[] = new int[100];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> Random rand = new Random(1000);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int stepCount = 0;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> public void operateStep() {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> stepCount++;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( stepCount%100 == 0 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> // enforce some tenuring</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange)+mutatingRange;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new UseLessWrapper(""+stepCount)));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( stepCount%200 == 199 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> // enforce some tenuring</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange)+mutatingRange*2;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new UseLessWrapper("a"+stepCount)));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( stepCount%400 == 299 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> // enforce some tenuring</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange)+mutatingRange*3;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new UseLessWrapper("a"+stepCount)));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( stepCount%1000 == 999 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> // enforce some tenuring</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * hmFillRange);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new UseLessWrapper("a"+stepCount)));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep/2; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new Dimension(key,key)));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep/8; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new UseLessWrapper(new UseLessWrapper(new UseLessWrapper(new UseLessWrapper("pok"+i))))));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep/16; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new int[50]));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep/32; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, ""+new UseLessWrapper(new int[100]));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep/32; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> Object[] wrapped = new Object[100];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for (int j = 0; j < wrapped.length; j++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> wrapped[j] = ""+j;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(wrapped));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < operationStep/64; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * mutatingRange /64);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new int[1000]));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < 4; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int key = (int) (rand.nextDouble() * 16);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(key, new UseLessWrapper(new byte[1000000]));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> public void fillMap() {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for ( int i = 0; i < hmFillRange; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> map.put(i, new UseLessWrapper(new UseLessWrapper(""+i)));</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> public void run() {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> fillMap();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> System.gc();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> System.out.println("static alloc " + (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / 1000 / 1000 + "mb");</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> long time = System.currentTimeMillis();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int count = 0;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> while ( (System.currentTimeMillis()-time) < runtime) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> count++;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> long tim = System.currentTimeMillis();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> operateStep();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int dur = (int) (System.currentTimeMillis()-tim);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( dur < 100 )</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> milliDelayCount[dur]++;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> else if ( dur < 10*100 )</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> hundredMilliDelayCount[dur/100]++;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> else {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> secondDelayCount[dur/1000]++;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> System.out.println("Iterations "+count);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> public void dumpResult() {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for (int i = 0; i < milliDelayCount.length; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int i1 = milliDelayCount[i];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( i1 > 0 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> System.out.println("["+i+"]\t"+i1);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for (int i = 0; i < hundredMilliDelayCount.length; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int i1 = hundredMilliDelayCount[i];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( i1 > 0 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> System.out.println("["+i*100+"]\t"+i1);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> for (int i = 0; i < secondDelayCount.length; i++) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int i1 = secondDelayCount[i];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> if ( i1 > 0 ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> System.out.println("["+i*1000+"]\t"+i1);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> int runtime = 60000 * 5;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> public static void main( String arg[] ) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> FSTGCMark fstgcMark = new FSTGCMark();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> fstgcMark.run();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> fstgcMark.dumpResult();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">}</span></div>
</div>
<div>
<br /></div>
Rüdiger Möllerhttp://www.blogger.com/profile/03711813786574992852noreply@blogger.com839