all repos — h3rald @ 94ae6a989b3b679d5d320bfdd67f81b16506e22d

The sources of https://h3rald.com

content/glyph/book/extending/internals.html

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
 100
 101
 102
 103
 104
 105
 106
 107
 108
 109
 110
 111
 112
 113
 114
 115
 116
 117
 118
 119
 120
 121
 122
 123
 124
 125
 126
 127
 128
 129
 130
 131
 132
 133
 134
 135
 136
 137
 138
 139
 140
 141
 142
 143
 144
 145
 146
 147
 148
 149
 150
 151
 152
 153
 154
 155
 156
 157
 158
 159
 160
 161
 162
 163
 164
 165
 166
 167
 168
 169
 170
 171
 172
 173
 174
 175
 176
 177
 178
 179
 180
 181
 182
 183
 184
 185
 186
 187
 188
 189
 190
 191
 192
 193
 194
 195
 196
 197
 198
 199
 200
 201
 202
 203
 204
<!DOCTYPE html> 
<html lang="en"> 
	<head> 
		<title>Glyph - A quick look at Glyph's internals</title> 



		<meta charset="utf-8" /> 
		<meta name="author" content="Fabio Cevasco" /> 
		<meta name="copyright" content="Fabio Cevasco" /> 
		<meta name="robots" content="all, follow" /> 
		<meta name="Revisit-After" content="2 Days" /> 
		<meta name="language" content="en" /> 
		<meta name="target_country" content="en-us" /> 
		<meta name="country" content="United States" /> 
		<meta name="description" content="H3RALD - Fabio Cevasco's Web Site" /> 

		<meta name="keywords" content="h3rald, fabio cevasco, glyph" /> 
		<link rel="shortcut icon" href="/favicon.png" type="image/png" /> 
		<meta content="44.388041;9.073248" name="ICBM" /> 


		<link rel="stylesheet" type="text/css" href="/styles/html5reset.css" /> 
		<link rel="stylesheet" type="text/css" href="/styles/style.css" /> 

		<!--[if lte IE 8]>
		<script src="/js/html5.js" type="text/javascript"></script>
		<![endif]--> 
	</head> 
	<body> 
		<!--[if lte IE 6]>
		<div id="ie-warning">
			This site is not compatible with Internet Explorer 6 or lower. You should consider using a more modern browser for a better &ndash; and <em>safer</em> &ndash; web experience. [<a href="http://browsehappy.com/browsers/">Read More &raquo;</a>]
		</div>
		<![endif]--> 

		<section id="container"> 

			<header class="page"> 
				<nav class="home-link"> 
					<a href="/"> 
						<img src="/images/h3rald_small.png" alt="H3RALD" class="default"/> 
						<![if !IE]> 
						<img src="/images/h3rald_hover_small.png" alt="H3RALD" class="hover"/> 
						<![endif]> 
					</a> 
				</nav> 

				<nav class="section"> 
					/<a href="/projects/" rel="archives">PROJECTS</a> 
				</nav> 

			</header> 
			<article class="page"> 
				<header> 
					<hgroup> 
					<h1>A quick look at Glyph's internals</h1> 
					</hgroup> 
				</header> 
				<section id="body-text" class="hyphenate"> 

							<nav><a href="/glyph/book/stats/links.html">Link Statistics &larr;</a><a href="/glyph/book/index.html">Contents</a><a href="/glyph/book/extending/macro_def.html">&rarr; Defining Custom Macros</a></nav>
								<p>If you plan on extending Glyph, knowing how it works inside helps. It is not mandatory by any means, but it definitely helps, especially when creating complex macros.</p>
	<p>What happens behind the scenes when you call <code>glyph compile</code>? Glyph's code is parsed, analyzed and then translated into text, and here's how:</p>
	<figure><img alt="-" src="images/glyph/document_generation.png" />
					<figcaption>A sequence diagram for document generation</figcaption></figure>
	<p>From the diagram, it is possible to divide the document generation process into three phases:</p>
<ul>
	<li>The <em>Parsing Phase</em> starts when a chunk of Glyph code is passed (by the <code>generate:document</code> Rake task, for example) to a <a href="http://yardoc.org/docs/h3rald-glyph/Glyph/Interpreter"><code>Glyph::Interpreter</code></a>. The interpreter initializes a <a href="http://yardoc.org/docs/h3rald-glyph/Glyph/Parser"><code>Glyph::Parser</code></a> that parses the code and returns an <em>Abstract Syntax Tree</em> (<span class="caps">AST</span>) of <a href="http://yardoc.org/docs/h3rald-glyph/Glyph/SyntaxNode"><code>Glyph::SyntaxNode</code></a> objects.</li>
	<li>The <em>Analysis Phase</em> (Processing) starts when the interpreter method calls the <code>analyze</code> method, instantiating a new <a href="http://yardoc.org/docs/h3rald-glyph/Glyph/Document"><code>Glyph::Document</code></a>. The <code>Glyph::Document</code> object evaluates the <span class="caps">AST</span> expanding all macro nodesth (that&#8217;s when macros are executed) and generates string.</li>
	<li>The <em>Finalization Phase</em> (Post-Processing) starts when the interpreter calls the <code>finalyze</code> method, causing the <code>Glyph::Document</code> object to perform a series of finalizations on the string obtained after analysis, i.e. it replaces escape sequences and placeholders.</li>
</ul>
	<section class="section">
<header><h1 id="h_73">Example: A short note</h1></header>
<p>As an example, consider the following Glyph code:</p>
		<div class="CodeRay">
  <div class="code"><pre><span class="no">1</span> fmi[something|#test]
<span class="no">2</span> ...
<span class="no">3</span> section[
<span class="no">4</span>   @title[Test Section]
<span class="no">5</span>   @id[test]
<span class="no">6</span> ...
<span class="no">7</span> ]</pre></div>
</div>

		<p>This simple snippet uses the <a href="/glyph/book/macros/macros_inline.html#m_fmi"><code>fmi</code></a> macro to link to a section later on in the document. When parsed, the produced AST is the following:</p>
		<div class="CodeRay">
  <div class="code"><pre><span class="no"> 1</span> {<span class="sy">:name</span>=&gt;<span class="sy"><span class="sy">:</span><span class="dl">&quot;</span><span class="k">--</span><span class="dl">&quot;</span></span>}
<span class="no"> 2</span>   {<span class="sy">:name</span>=&gt;<span class="sy">:fmi</span>, <span class="sy">:escape</span>=&gt;<span class="pc">false</span>}
<span class="no"> 3</span>     {<span class="sy">:name</span>=&gt;<span class="sy"><span class="sy">:</span><span class="dl">&quot;</span><span class="k">0</span><span class="dl">&quot;</span></span>}
<span class="no"> 4</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="k">something</span><span class="dl">&quot;</span></span>}
<span class="no"> 5</span>     {<span class="sy">:name</span>=&gt;<span class="sy"><span class="sy">:</span><span class="dl">&quot;</span><span class="k">1</span><span class="dl">&quot;</span></span>}
<span class="no"> 6</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="k">#test</span><span class="dl">&quot;</span></span>}
<span class="no"> 7</span>   {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\n</span><span class="dl">&quot;</span></span>}
<span class="no"> 8</span>   {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\</span><span class="k">[</span><span class="dl">&quot;</span></span>, <span class="sy">:escaped</span>=&gt;<span class="pc">true</span>}
<span class="no"> 9</span>   {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="k">...</span><span class="dl">&quot;</span></span>}
<span class="no"><strong>10</strong></span>   {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\</span><span class="k">]</span><span class="dl">&quot;</span></span>, <span class="sy">:escaped</span>=&gt;<span class="pc">true</span>}
<span class="no">11</span>   {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\n</span><span class="dl">&quot;</span></span>}
<span class="no">12</span>   {<span class="sy">:name</span>=&gt;<span class="sy">:section</span>, <span class="sy">:escape</span>=&gt;<span class="pc">false</span>}
<span class="no">13</span>     {<span class="sy">:name</span>=&gt;<span class="sy"><span class="sy">:</span><span class="dl">&quot;</span><span class="k">0</span><span class="dl">&quot;</span></span>}
<span class="no">14</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\n</span><span class="ch">\t</span><span class="dl">&quot;</span></span>}
<span class="no">15</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\n</span><span class="ch">\t</span><span class="dl">&quot;</span></span>}
<span class="no">16</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\n</span><span class="dl">&quot;</span></span>}
<span class="no">17</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\</span><span class="k">[</span><span class="dl">&quot;</span></span>, <span class="sy">:escaped</span>=&gt;<span class="pc">true</span>}
<span class="no">18</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="k">...</span><span class="dl">&quot;</span></span>}
<span class="no">19</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\</span><span class="k">]</span><span class="dl">&quot;</span></span>, <span class="sy">:escaped</span>=&gt;<span class="pc">true</span>}
<span class="no"><strong>20</strong></span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="ch">\n</span><span class="dl">&quot;</span></span>}
<span class="no">21</span>     {<span class="sy">:name</span>=&gt;<span class="sy">:title</span>, <span class="sy">:escape</span>=&gt;<span class="pc">false</span>}
<span class="no">22</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="k">Test Section</span><span class="dl">&quot;</span></span>}
<span class="no">23</span>     {<span class="sy">:name</span>=&gt;<span class="sy">:id</span>, <span class="sy">:escape</span>=&gt;<span class="pc">false</span>}
<span class="no">24</span>       {<span class="sy">:value</span>=&gt;<span class="s"><span class="dl">&quot;</span><span class="k">test</span><span class="dl">&quot;</span></span>}</pre></div>
</div>

		<p>This output is produced by calling the <code>inspect</code> method on the AST. Each <a href="http://yardoc.org/docs/h3rald-glyph/Glyph/SyntaxNode"><code>Glyph::SyntaxNode</code></a> object in the tree is basically an ordinary Glyph Hash with a parent and 0 or more chidren, so the code snippets above shows how the syntax nodes are nested.</p>
		<p>The AST contains information about macro, parameter and attribute names, and escaping, and raw text values (the nodes without a <code>:name</code> key), but nothing more.</p>
		<p>When the AST is analyzed, the resulting textual output is the following:</p>
		<div class="CodeRay">
  <div class="code"><pre><span class="no">1</span> <span class="ta">&lt;span</span> <span class="an">class</span>=<span class="s"><span class="dl">&quot;</span><span class="k">fmi</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>for more information on something, see ‡‡‡‡‡PLACEHOLDER ¤ 1‡‡‡‡‡
<span class="no">2</span> <span class="ta">&lt;/span&gt;</span>
<span class="no">3</span> \.[...\.]
<span class="no">4</span> <span class="ta">&lt;div</span> <span class="an">class</span>=<span class="s"><span class="dl">&quot;</span><span class="k">section</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>
<span class="no">5</span> <span class="ta">&lt;h2</span> <span class="an">id</span>=<span class="s"><span class="dl">&quot;</span><span class="k">test</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>Test Section<span class="ta">&lt;/h2&gt;</span>
<span class="no">6</span> \.[...\.]
<span class="no">7</span> 
<span class="no">8</span> <span class="ta">&lt;/div&gt;</span></pre></div>
</div>

		<p>This looks almost perfect, except that:</p>
		<ul><li>There's a nasty placeholder instead of a link: this is due to the fact that when the link is processed, there is no <code>#text</code> anchor in the document, but there may be one afterwards (and there will be).</li>
			<li>There are some escaped brackets.</li></ul>
		<p>Finally, when the document is finalized, placeholders and escape sequences are removed and the final result is the following:</p>
		<div class="CodeRay">
  <div class="code"><pre><span class="no">1</span> <span class="ta">&lt;span</span> <span class="an">class</span>=<span class="s"><span class="dl">&quot;</span><span class="k">fmi</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>for more information on something, 
<span class="no">2</span>   see <span class="ta">&lt;a</span> <span class="an">href</span>=<span class="s"><span class="dl">&quot;</span><span class="k">#test</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>Test Section<span class="ta">&lt;/a&gt;</span><span class="ta">&lt;/span&gt;</span>
<span class="no">3</span> [...]
<span class="no">4</span> <span class="ta">&lt;div</span> <span class="an">class</span>=<span class="s"><span class="dl">&quot;</span><span class="k">section</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>
<span class="no">5</span> <span class="ta">&lt;h2</span> <span class="an">id</span>=<span class="s"><span class="dl">&quot;</span><span class="k">test</span><span class="dl">&quot;</span></span><span class="ta">&gt;</span>Test Section<span class="ta">&lt;/h2&gt;</span>
<span class="no">6</span> [...]
<span class="no">7</span> 
<span class="no">8</span> <span class="ta">&lt;/div&gt;</span></pre></div>
</div>

</section>
							<nav><a href="/glyph/book/stats/links.html">Link Statistics &larr;</a><a href="/glyph/book/index.html">Contents</a><a href="/glyph/book/extending/macro_def.html">&rarr; Defining Custom Macros</a></nav>
				</section> 
			</article> 


			<footer> 
				<section class="ads"> 
					<script type="text/javascript"><!--
						google_ad_client = "pub-2871497824158668";
						/* 728x90, created 9/10/10 */
						google_ad_slot = "3963343166";
						google_ad_width = 728;
						google_ad_height = 90;
						//-->
					</script> 
					<script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> 
					</script> 
				</section> 
				<section> 
					<nav> 
						<a href="/about/">ABOUT</a>|<a href="/contact/">CONTACT</a> 
					</nav> 
					<p>H3RALD Web Site v8.1 &mdash; &copy; 2010 &mdash; <em>Fabio Cevasco</em></p> 
				</section> 
			</footer> 
		</section><!-- #container end --> 
		<script src="http://www.google.com/jsapi?key=ABQIAAAAr6RY1Z6dchG_sX9WDLSy3xRlq2n1sm52B5HDRR5tm6o8XM18FhR56xHNNH6CsX86uN5VoTrglpyOyQ" type="text/javascript"></script> 
		<!-- <script src="/js/jquery.js" type="text/javascript"></script> --> 
		<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script> 
		<script src="/js/jquery-timeago.js" type="text/javascript"></script> 
		<script src="/js/jquery-easing.js" type="text/javascript"></script> 
		<script src="/js/jquery-fancybox.js" type="text/javascript"></script> 
		<script src="/js/jquery-toc.js" type="text/javascript"></script> 
		<script src="/js/date.js" type="text/javascript"></script> 
		<script src="/js/feeds.js" type="text/javascript"></script> 
		<script src="/js/search.js" type="text/javascript"></script> 
		<script src="/js/hyphenator.min.js" type="text/javascript"></script> 
		<script src="/js/init.js" type="text/javascript"></script> 
		<!-- Start Google Analytics --> 
		<script type="text/javascript"> 
			var _gaq = _gaq || [];
			_gaq.push(['_setAccount', 'UA-18587377-1']);
			_gaq.push(['_trackPageview']);

			(function() {
			var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
			ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
			var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
			})();
		</script> 
		<!-- End Google Analytics --> 
		<!-- Start of StatCounter Code --> 
		<script type="text/javascript"> 
			var sc_project=6193656; 
			var sc_invisible=1; 
			var sc_security="57f7ee2a"; 
		</script> 
		<script type="text/javascript" src="http://www.statcounter.com/counter/counter_xhtml.js"></script> 
		<!-- End of StatCounter Code --> 
	</body> 
</html>