content/glyph/book/extending/internals.html
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 |
<!DOCTYPE html>
<html lang="en">
<head>
<title>A quick look at Glyph's internals - Glyph</title>
<link href='http://fonts.googleapis.com/css?family=Crimson+Text:400,400italic,600,600italic,700,700italic' rel='stylesheet' type='text/css'>
<meta charset="utf-8" />
<meta name="author" content="Fabio Cevasco" />
<meta name="robots" content="all, follow" />
<meta name="Revisit-After" content="2 Days" />
<meta name="description" content="H3RALD - Fabio Cevasco's Web Site" />
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="keywords" content="h3rald, fabio cevasco, glyph, ruby, opensource" />
<link rel="shortcut icon" href="/favicon.png" type="image/png" />
<meta content="44.388041;9.073248" name="ICBM" />
<!-- <link rel="stylesheet" type="text/css" href="/styles/style.css" /> -->
<!--[if lt IE 9]>
<script src="http://html5shiv.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
<link rel="stylesheet" type="text/css" href="/styles/bootstrap.css" />
<link rel="stylesheet" type="text/css" href="/styles/style.css" />
<script src="http://www.google.com/jsapi?key=ABQIAAAAr6RY1Z6dchG_sX9WDLSy3xRlq2n1sm52B5HDRR5tm6o8XM18FhR56xHNNH6CsX86uN5VoTrglpyOyQ" type="text/javascript"></script>
<script src="/js/sevenup.js" type="text/javascript"></script>
<script src="/js/sevenup_black.js" type="text/javascript"></script>
<script src="http://code.jquery.com/jquery-latest.js" type="text/javascript"></script>
<script src="/js/jquery-timeago.js" type="text/javascript"></script>
<script src="/js/date.js" type="text/javascript"></script>
<script src="/js/feeds.js" type="text/javascript"></script>
<script src="/js/hyphenator.js" type="text/javascript"></script>
<script src="/js/init.js" type="text/javascript"></script>
<script src="/js/search.js" type="text/javascript"></script>
<script src="/js/bootstrap.js" type="text/javascript"></script>
</head>
<body>
<div class="container">
<div class="row" id="header-row">
<div class="span6 main-title">
<h1>
<a href="/"><img src="/img/h3rald.png" alt="H3RALD" /></a>
</h1>
</div>
<div class="span4 offset2">
<div class="gcse-searchbox" data-enableAutoComplete="true"></div>
</div>
</div>
<div id="search_results">
<div class="gcse-searchresults" data-linkTarget="_top"></div>
</div>
<div id=main-navbar class="navbar navbar-inverse">
<div class="navbar-inner">
<button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<div class="nav-collapse collapse">
<ul class="nav">
<li><a href="/">HOME</a></li>
<li><a href="/projects/">PROJECTS</a></li>
<li><a href="/tags/">TAGS</a></li>
<li><a href="/archives/">ARCHIVES</a></li>
<li><a href="/about/">ABOUT</a></li>
<li><a href="/contact/">CONTACT</a></li>
</ul>
</div>
</div>
</div>
</div>
<div class="container">
<article class="page">
<header class="row">
<hgroup class="span9">
<h1>A quick look at Glyph's internals</h1>
<h2>Glyph</h2>
</hgroup>
</header>
<section id="body-text" class="hyphenate">
<nav class="navigation"><a href="/glyph/book/stats/links.html">← Link Statistics</a> | <a href="/glyph/book/index.html">Contents</a> | <a href="/glyph/book/extending/macro_def.html">Defining Custom Macros →</a></nav>
<p>If you plan on extending Glyph, knowing how it works inside helps. It is not mandatory by any means, but it definitely helps, especially when creating complex macros.</p>
<p>What happens behind the scenes when you call <code>glyph compile</code>? Glyph's code is parsed, analyzed and then translated into text, and here's how:</p>
<figure><img src="/glyph/book/images/glyph/document_generation.png" /><figcaption>A sequence diagram for document generation</figcaption></figure>
<p>From the diagram, it is possible to divide the document generation process into three phases:</p>
<ul>
<li>The <em>Parsing Phase</em> starts when a chunk of Glyph code is passed (by the <code>generate:document</code> Rake task, for example) to a <a href="http://rubydoc.info/gems/glyph/Glyph/Interpreter"><code>Glyph::Interpreter</code></a>. The interpreter initializes a <a href="http://rubydoc.info/gems/glyph/Glyph/Parser"><code>Glyph::Parser</code></a> that parses the code and returns an <em>Abstract Syntax Tree</em> (<span class="caps">AST</span>) of <a href="http://rubydoc.info/gems/glyph/Glyph/SyntaxNode"><code>Glyph::SyntaxNode</code></a> objects.</li>
<li>The <em>Analysis Phase</em> (Processing) starts when the interpreter method calls the <code>analyze</code> method, instantiating a new <a href="http://rubydoc.info/gems/glyph/Glyph/Document"><code>Glyph::Document</code></a>. The <code>Glyph::Document</code> object evaluates the <span class="caps">AST</span> expanding all macro nodesth (that’s when macros are executed) and generates string.</li>
<li>The <em>Finalization Phase</em> (Post-Processing) starts when the interpreter calls the <code>finalyze</code> method, causing the <code>Glyph::Document</code> object to perform a series of finalizations on the string obtained after analysis, i.e. it replaces escape sequences and placeholders.</li>
</ul>
<section class="section">
<header><h1 id="h_83" class="toc">Example: A short note</h1></header>
<p>As an example, consider the following Glyph code:</p>
<div class="CodeRay">
<div class="code"><pre><span class="line-numbers"><a href="#n1" name="n1">1</a></span>fmi[something|#test]
<span class="line-numbers"><a href="#n2" name="n2">2</a></span>...
<span class="line-numbers"><a href="#n3" name="n3">3</a></span>section[
<span class="line-numbers"><a href="#n4" name="n4">4</a></span> @title[Test Section]
<span class="line-numbers"><a href="#n5" name="n5">5</a></span> @id[test]
<span class="line-numbers"><a href="#n6" name="n6">6</a></span>...
<span class="line-numbers"><a href="#n7" name="n7">7</a></span>]</pre></div>
</div>
<p>This simple snippet uses the <a href="/glyph/book/macros/macros_inline.html#m_fmi"><code>fmi</code></a> macro to link to a section later on in the document. When parsed, the produced AST is the following:</p>
<div class="CodeRay">
<div class="code"><pre><span class="line-numbers"> <a href="#n1" name="n1">1</a></span>{<span class="symbol">:name</span>=><span class="symbol"><span class="symbol">:</span><span class="delimiter">"</span><span class="content">--</span><span class="delimiter">"</span></span>}
<span class="line-numbers"> <a href="#n2" name="n2">2</a></span> {<span class="symbol">:name</span>=><span class="symbol">:fmi</span>, <span class="symbol">:escape</span>=><span class="predefined-constant">false</span>}
<span class="line-numbers"> <a href="#n3" name="n3">3</a></span> {<span class="symbol">:name</span>=><span class="symbol"><span class="symbol">:</span><span class="delimiter">"</span><span class="content">0</span><span class="delimiter">"</span></span>}
<span class="line-numbers"> <a href="#n4" name="n4">4</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="content">something</span><span class="delimiter">"</span></span>}
<span class="line-numbers"> <a href="#n5" name="n5">5</a></span> {<span class="symbol">:name</span>=><span class="symbol"><span class="symbol">:</span><span class="delimiter">"</span><span class="content">1</span><span class="delimiter">"</span></span>}
<span class="line-numbers"> <a href="#n6" name="n6">6</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="content">#test</span><span class="delimiter">"</span></span>}
<span class="line-numbers"> <a href="#n7" name="n7">7</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\n</span><span class="delimiter">"</span></span>}
<span class="line-numbers"> <a href="#n8" name="n8">8</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\</span><span class="content">[</span><span class="delimiter">"</span></span>, <span class="symbol">:escaped</span>=><span class="predefined-constant">true</span>}
<span class="line-numbers"> <a href="#n9" name="n9">9</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="content">...</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><strong><a href="#n10" name="n10">10</a></strong></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\</span><span class="content">]</span><span class="delimiter">"</span></span>, <span class="symbol">:escaped</span>=><span class="predefined-constant">true</span>}
<span class="line-numbers"><a href="#n11" name="n11">11</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\n</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n12" name="n12">12</a></span> {<span class="symbol">:name</span>=><span class="symbol">:section</span>, <span class="symbol">:escape</span>=><span class="predefined-constant">false</span>}
<span class="line-numbers"><a href="#n13" name="n13">13</a></span> {<span class="symbol">:name</span>=><span class="symbol"><span class="symbol">:</span><span class="delimiter">"</span><span class="content">0</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n14" name="n14">14</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\n</span><span class="char">\t</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n15" name="n15">15</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\n</span><span class="char">\t</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n16" name="n16">16</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\n</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n17" name="n17">17</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\</span><span class="content">[</span><span class="delimiter">"</span></span>, <span class="symbol">:escaped</span>=><span class="predefined-constant">true</span>}
<span class="line-numbers"><a href="#n18" name="n18">18</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="content">...</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n19" name="n19">19</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\</span><span class="content">]</span><span class="delimiter">"</span></span>, <span class="symbol">:escaped</span>=><span class="predefined-constant">true</span>}
<span class="line-numbers"><strong><a href="#n20" name="n20">20</a></strong></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="char">\n</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n21" name="n21">21</a></span> {<span class="symbol">:name</span>=><span class="symbol">:title</span>, <span class="symbol">:escape</span>=><span class="predefined-constant">false</span>}
<span class="line-numbers"><a href="#n22" name="n22">22</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="content">Test Section</span><span class="delimiter">"</span></span>}
<span class="line-numbers"><a href="#n23" name="n23">23</a></span> {<span class="symbol">:name</span>=><span class="symbol">:id</span>, <span class="symbol">:escape</span>=><span class="predefined-constant">false</span>}
<span class="line-numbers"><a href="#n24" name="n24">24</a></span> {<span class="symbol">:value</span>=><span class="string"><span class="delimiter">"</span><span class="content">test</span><span class="delimiter">"</span></span>}</pre></div>
</div>
<p>This output is produced by calling the <code>inspect</code> method on the AST. Each <a href="http://rubydoc.info/gems/glyph/Glyph/SyntaxNode"><code>Glyph::SyntaxNode</code></a> object in the tree is basically an ordinary Glyph Hash with a parent and 0 or more chidren, so the code snippets above shows how the syntax nodes are nested.</p>
<p>The AST contains information about macro, parameter and attribute names, and escaping, and raw text values (the nodes without a <code>:name</code> key), but nothing more.</p>
<p>When the AST is analyzed, the resulting textual output is the following:</p>
<div class="CodeRay">
<div class="code"><pre><span class="line-numbers"><a href="#n1" name="n1">1</a></span><span class="tag"><span</span> <span class="attribute-name">class</span>=<span class="string"><span class="delimiter">"</span><span class="content">fmi</span><span class="delimiter">"</span></span><span class="tag">></span>for more information on something, see ‡‡‡‡‡PLACEHOLDER ¤ 1‡‡‡‡‡
<span class="line-numbers"><a href="#n2" name="n2">2</a></span><span class="tag"></span></span>
<span class="line-numbers"><a href="#n3" name="n3">3</a></span>\/[...\/]
<span class="line-numbers"><a href="#n4" name="n4">4</a></span><span class="tag"><div</span> <span class="attribute-name">class</span>=<span class="string"><span class="delimiter">"</span><span class="content">section</span><span class="delimiter">"</span></span><span class="tag">></span>
<span class="line-numbers"><a href="#n5" name="n5">5</a></span><span class="tag"><h2</span> <span class="attribute-name">id</span>=<span class="string"><span class="delimiter">"</span><span class="content">test</span><span class="delimiter">"</span></span><span class="tag">></span>Test Section<span class="tag"></h2></span>
<span class="line-numbers"><a href="#n6" name="n6">6</a></span>\/[...\/]
<span class="line-numbers"><a href="#n7" name="n7">7</a></span>
<span class="line-numbers"><a href="#n8" name="n8">8</a></span><span class="tag"></div></span></pre></div>
</div>
<p>This looks almost perfect, except that:</p>
<ul>
<li>There's a nasty placeholder instead of a link: this is due to the fact that when the link is processed, there is no <code>#text</code> anchor in the document, but there may be one afterwards (and there will be).</li>
<li>There are some escaped brackets.</li>
</ul>
<p>Finally, when the document is finalized, placeholders and escape sequences are removed and the final result is the following:</p>
<div class="CodeRay">
<div class="code"><pre><span class="line-numbers"><a href="#n1" name="n1">1</a></span><span class="tag"><span</span> <span class="attribute-name">class</span>=<span class="string"><span class="delimiter">"</span><span class="content">fmi</span><span class="delimiter">"</span></span><span class="tag">></span>for more information on something,
<span class="line-numbers"><a href="#n2" name="n2">2</a></span> see <span class="tag"><a</span> <span class="attribute-name">href</span>=<span class="string"><span class="delimiter">"</span><span class="content">#test</span><span class="delimiter">"</span></span><span class="tag">></span>Test Section<span class="tag"></a></span><span class="tag"></span></span>
<span class="line-numbers"><a href="#n3" name="n3">3</a></span>[...]
<span class="line-numbers"><a href="#n4" name="n4">4</a></span><span class="tag"><div</span> <span class="attribute-name">class</span>=<span class="string"><span class="delimiter">"</span><span class="content">section</span><span class="delimiter">"</span></span><span class="tag">></span>
<span class="line-numbers"><a href="#n5" name="n5">5</a></span><span class="tag"><h2</span> <span class="attribute-name">id</span>=<span class="string"><span class="delimiter">"</span><span class="content">test</span><span class="delimiter">"</span></span><span class="tag">></span>Test Section<span class="tag"></h2></span>
<span class="line-numbers"><a href="#n6" name="n6">6</a></span>[...]
<span class="line-numbers"><a href="#n7" name="n7">7</a></span>
<span class="line-numbers"><a href="#n8" name="n8">8</a></span><span class="tag"></div></span></pre></div>
</div>
</section>
<nav class="navigation"><a href="/glyph/book/stats/links.html">← Link Statistics</a> | <a href="/glyph/book/index.html">Contents</a> | <a href="/glyph/book/extending/macro_def.html">Defining Custom Macros →</a></nav>
</section>
</article>
</div>
<footer>
<section>
<p>H3RALD Web Site v8.3 © 2004 — 2013 <em>Fabio Cevasco</em></p>
</section>
</footer>
<!-- Start Google Analytics -->
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-18587377-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
<!-- End Google Analytics -->
<!-- Start of StatCounter Code -->
<script type="text/javascript">
var sc_project=6193656;
var sc_invisible=1;
var sc_security="57f7ee2a";
</script>
<script type="text/javascript" src="http://www.statcounter.com/counter/counter_xhtml.js"></script>
<!-- End of StatCounter Code -->
</body>
</html>
|