<!DOCTYPE html> | |
<html lang="en"> | |
<head> | |
<meta charset="UTF-8"> | |
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]--> | |
<meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
<meta name="generator" content="Asciidoctor 1.5.4"> | |
<title>odb User Guide</title> | |
<style> | |
/** | |
* @@@ START COPYRIGHT @@@ | |
* | |
* Licensed to the Apache Software Foundation (ASF) under one | |
* or more contributor license agreements. See the NOTICE file | |
* distributed with this work for additional information | |
* regarding copyright ownership. The ASF licenses this file | |
* to you under the Apache License, Version 2.0 (the | |
* "License"); you may not use this file except in compliance | |
* with the License. You may obtain a copy of the License at | |
* | |
* http://www.apache.org/licenses/LICENSE-2.0 | |
* | |
* Unless required by applicable law or agreed to in writing, | |
* software distributed under the License is distributed on an | |
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | |
* KIND, either express or implied. See the License for the | |
* specific language governing permissions and limitations | |
* under the License. | |
* | |
* @@@ END COPYRIGHT @@@ | |
*/ | |
/* Asciidoctor default stylesheet | MIT License | http://asciidoctor.org */ | |
/* Remove the comments around the @import statement below when using this as a custom stylesheet */ | |
/*@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400";*/ | |
article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section,summary{display:block} | |
audio,canvas,video{display:inline-block} | |
audio:not([controls]){display:none;height:0} | |
[hidden],template{display:none} | |
script{display:none!important} | |
html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%} | |
body{margin:0} | |
a{background:transparent} | |
a:focus{outline:thin dotted} | |
a:active,a:hover{outline:0} | |
h1{font-size:2em;margin:.67em 0} | |
abbr[title]{border-bottom:1px dotted} | |
b,strong{font-weight:bold} | |
dfn{font-style:italic} | |
hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0} | |
mark{background:#ff0;color:#000} | |
code,kbd,pre,samp{font-family:monospace;font-size:1em} | |
pre{white-space:pre-wrap} | |
q{quotes:"\201C" "\201D" "\2018" "\2019"} | |
small{font-size:80%} | |
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline} | |
sup{top:-.5em} | |
sub{bottom:-.25em} | |
img{border:0} | |
svg:not(:root){overflow:hidden} | |
figure{margin:0} | |
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em} | |
legend{border:0;padding:0} | |
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0} | |
button,input{line-height:normal} | |
button,select{text-transform:none} | |
button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer} | |
button[disabled],html input[disabled]{cursor:default} | |
input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0} | |
input[type="search"]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box} | |
input[type="search"]::-webkit-search-cancel-button,input[type="search"]::-webkit-search-decoration{-webkit-appearance:none} | |
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0} | |
textarea{overflow:auto;vertical-align:top} | |
table{border-collapse:collapse;border-spacing:0} | |
*,*:before,*:after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box} | |
html,body{font-size:100%} | |
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif,serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto} | |
a:hover{cursor:pointer} | |
img,object,embed{max-width:100%;height:auto} | |
object,embed{height:100%} | |
img{-ms-interpolation-mode:bicubic} | |
#map_canvas img,#map_canvas embed,#map_canvas object,.map_canvas img,.map_canvas embed,.map_canvas object{max-width:none!important} | |
.left{float:left!important} | |
.right{float:right!important} | |
.text-left{text-align:left!important} | |
.text-right{text-align:right!important} | |
.text-center{text-align:center!important} | |
.text-justify{text-align:justify!important} | |
.hide{display:none} | |
.antialiased,body{-webkit-font-smoothing:antialiased} | |
img{display:inline-block;vertical-align:middle} | |
textarea{height:auto;min-height:50px} | |
select{width:100%} | |
p.lead,.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{font-size:1.21875em;line-height:1.6} | |
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#3188ac;font-weight:400;margin-top:0;margin-bottom:.25em} | |
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr} | |
a{color:#2156a5;text-decoration:underline;line-height:inherit} | |
a:hover,a:focus{color:#1d4b8f} | |
a img{border:none} | |
p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility} | |
p aside{font-size:.875em;line-height:1.35;font-style:italic} | |
/* Defines headings */ | |
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-weight:bold;font-style:normal;color:#3188ac;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em} | |
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0} | |
h1{font-size:2.125em} | |
h2{font-size:1.6875em} | |
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em} | |
h4,h5{font-size:1.125em} | |
h6{font-size:1em} | |
hr{border:solid #ddddd8;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0} | |
em,i{font-style:italic;line-height:inherit} | |
strong,b{font-weight:bold;line-height:inherit} | |
small{font-size:60%;line-height:inherit} | |
/* Defines the `text` (passthru) format */ | |
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:bold;color:#3188ac} | |
ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit} | |
ul,ol,ul.no-bullet,ol.no-bullet{margin-left:1.5em} | |
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em} | |
ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit} | |
ul.square{list-style-type:square} | |
ul.circle{list-style-type:circle} | |
ul.disc{list-style-type:disc} | |
ul.no-bullet{list-style:none} | |
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0} | |
dl dt{margin-bottom:.3125em;font-weight:bold} | |
dl dd{margin-bottom:1.25em} | |
abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help} | |
abbr{text-transform:none} | |
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd} | |
blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)} | |
blockquote cite:before{content:"\2014 \0020"} | |
blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)} | |
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)} | |
@media only screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2} | |
h1{font-size:2.75em} | |
h2{font-size:2.3125em} | |
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em} | |
h4{font-size:1.4375em}}table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede} | |
table thead,table tfoot{background:#f7f8f7;font-weight:bold} | |
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left} | |
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)} | |
table tr.even,table tr.alt,table tr:nth-of-type(even){background:#f8f8f7} | |
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6} | |
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em} | |
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400} | |
.clearfix:before,.clearfix:after,.float-group:before,.float-group:after{content:" ";display:table} | |
.clearfix:after,.float-group:after{clear:both} | |
*:not(pre)>code{font-size:1.0em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed} | |
/**:not(pre)>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background-color:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}*/ | |
pre,pre>code{line-height:1.45;color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;text-rendering:optimizeSpeed} | |
.keyseq{color:rgba(51,51,51,.8)} | |
kbd{display:inline-block;color:rgba(0,0,0,.8);font-size:.75em;line-height:1.4;background-color:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:-.15em .15em 0 .15em;padding:.2em .6em .2em .5em;vertical-align:middle;white-space:nowrap} | |
.keyseq kbd:first-child{margin-left:0} | |
.keyseq kbd:last-child{margin-right:0} | |
.menuseq,.menu{color:rgba(0,0,0,.8)} | |
b.button:before,b.button:after{position:relative;top:-1px;font-weight:400} | |
b.button:before{content:"[";padding:0 3px 0 2px} | |
b.button:after{content:"]";padding:0 2px 0 3px} | |
p a>code:hover{color:rgba(0,0,0,.9)} | |
#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em} | |
#header:before,#header:after,#content:before,#content:after,#footnotes:before,#footnotes:after,#footer:before,#footer:after{content:" ";display:table} | |
#header:after,#content:after,#footnotes:after,#footer:after{clear:both} | |
#content{margin-top:1.25em} | |
#content:before{content:none} | |
/* #header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0} */ | |
#header>h1:first-child{color:#3188ac;margin-top:2.25rem;margin-bottom:0} | |
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #ddddd8} | |
#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #ddddd8;padding-bottom:8px} | |
#header .details{border-bottom:1px solid #ddddd8;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap} | |
#header .details span:first-child{margin-left:-.125em} | |
#header .details span.email a{color:rgba(0,0,0,.85)} | |
#header .details br{display:none} | |
#header .details br+span:before{content:"\00a0\2013\00a0"} | |
#header .details br+span.author:before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)} | |
#header .details br+span#revremark:before{content:"\00a0|\00a0"} | |
#header #revnumber{text-transform:capitalize} | |
#header #revnumber:after{content:"\00a0"} | |
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #ddddd8;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem} | |
#toc{border-bottom:1px solid #efefed;padding-bottom:.5em} | |
#toc>ul{margin-left:.125em} | |
#toc ul.sectlevel0>li>a{font-style:italic} | |
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0} | |
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none} | |
#toc a{text-decoration:none} | |
#toc a:active{text-decoration:underline} | |
#toctitle{color:#3188ac;font-size:1.2em} | |
@media only screen and (min-width:768px){#toctitle{font-size:1.375em} | |
body.toc2{padding-left:15em;padding-right:0} | |
#toc.toc2{margin-top:0!important;background-color:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #efefed;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto} | |
#toc.toc2 #toctitle{margin-top:0;font-size:1.2em} | |
#toc.toc2>ul{font-size:.9em;margin-bottom:0} | |
#toc.toc2 ul ul{margin-left:0;padding-left:1em} | |
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em} | |
body.toc2.toc-right{padding-left:0;padding-right:15em} | |
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #efefed;left:auto;right:0}}@media only screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0} | |
/* Controls width of panel */ | |
#toc.toc2{width:20em} | |
#toc.toc2 #toctitle{font-size:1.375em} | |
#toc.toc2>ul{font-size:.95em} | |
#toc.toc2 ul ul{padding-left:1.25em} | |
body.toc2.toc-right{padding-left:0;padding-right:20em}}#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} | |
#content #toc>:first-child{margin-top:0} | |
#content #toc>:last-child{margin-bottom:0} | |
#footer{max-width:100%;background-color:rgba(0,0,0,.8);padding:1.25em} | |
#footer-text,#footer_nav{color:rgba(255,255,255,.8);line-height:1.44} | |
#footer a{color: #990000} | |
.sect1{padding-bottom:.625em} | |
@media only screen and (min-width:768px){.sect1{padding-bottom:1.25em}}.sect1+.sect1{border-top:1px solid #efefed} | |
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400} | |
#content h1>a.anchor:before,h2>a.anchor:before,h3>a.anchor:before,#toctitle>a.anchor:before,.sidebarblock>.content>.title>a.anchor:before,h4>a.anchor:before,h5>a.anchor:before,h6>a.anchor:before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em} | |
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible} | |
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#990000;text-decoration:none} | |
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221} | |
.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em} | |
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif,serif;font-size:1rem;font-weight:bold} | |
/* Here */ | |
table.tableblock>caption.title{white-space:nowrap;overflow:visible;max-width:0} | |
.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{color:rgba(0,0,0,.85)} | |
table.tableblock #preamble>.sectionbody>.paragraph:first-of-type p{font-size:inherit} | |
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%} | |
.admonitionblock>table td.icon{text-align:center;width:80px} | |
.admonitionblock>table td.icon img{max-width:none} | |
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase} | |
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #ddddd8;color:rgba(0,0,0,.6)} | |
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0} | |
.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px} | |
.exampleblock>.content>:first-child{margin-top:0} | |
.exampleblock>.content>:last-child{margin-bottom:0} | |
.sidebarblock{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} | |
.sidebarblock>:first-child{margin-top:0} | |
.sidebarblock>:last-child{margin-bottom:0} | |
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center} | |
.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0} | |
.literalblock pre,.listingblock pre:not(.highlight),.listingblock pre[class="highlight"],.listingblock pre[class^="highlight "],.listingblock pre.CodeRay,.listingblock pre.prettyprint{background:#f7f7f8} | |
.sidebarblock .literalblock pre,.sidebarblock .listingblock pre:not(.highlight),.sidebarblock .listingblock pre[class="highlight"],.sidebarblock .listingblock pre[class^="highlight "],.sidebarblock .listingblock pre.CodeRay,.sidebarblock .listingblock pre.prettyprint{background:#f2f1f1} | |
.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;padding:1em;font-size:.8125em} | |
.literalblock pre.nowrap,.literalblock pre[class].nowrap,.listingblock pre.nowrap,.listingblock pre[class].nowrap{overflow-x:auto;white-space:pre;word-wrap:normal} | |
@media only screen and (min-width:768px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:.90625em}}@media only screen and (min-width:1280px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:1em}}.literalblock.output pre{color:#f7f7f8;background-color:rgba(0,0,0,.9)} | |
.listingblock pre.highlightjs{padding:0} | |
.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px} | |
.listingblock pre.prettyprint{border-width:0} | |
.listingblock>.content{position:relative} | |
.listingblock code[data-lang]:before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:#999} | |
.listingblock:hover code[data-lang]:before{display:block} | |
.listingblock.terminal pre .command:before{content:attr(data-prompt);padding-right:.5em;color:#999} | |
.listingblock.terminal pre .command:not([data-prompt]):before{content:"$"} | |
table.pyhltable{border-collapse:separate;border:0;margin-bottom:0;background:none} | |
table.pyhltable td{vertical-align:top;padding-top:0;padding-bottom:0} | |
table.pyhltable td.code{padding-left:.75em;padding-right:0} | |
pre.pygments .lineno,table.pyhltable td:not(.code){color:#999;padding-left:0;padding-right:.5em;border-right:1px solid #ddddd8} | |
pre.pygments .lineno{display:inline-block;margin-right:.25em} | |
table.pyhltable .linenodiv{background:none!important;padding-right:0!important} | |
.quoteblock{margin:0 1em 1.25em 1.5em;display:table} | |
.quoteblock>.title{margin-left:-1.5em;margin-bottom:.75em} | |
.quoteblock blockquote,.quoteblock blockquote p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify} | |
.quoteblock blockquote{margin:0;padding:0;border:0} | |
.quoteblock blockquote:before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)} | |
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0} | |
.quoteblock .attribution{margin-top:.5em;margin-right:.5ex;text-align:right} | |
.quoteblock .quoteblock{margin-left:0;margin-right:0;padding:.5em 0;border-left:3px solid rgba(0,0,0,.6)} | |
.quoteblock .quoteblock blockquote{padding:0 0 0 .75em} | |
.quoteblock .quoteblock blockquote:before{display:none} | |
.verseblock{margin:0 1em 1.25em 1em} | |
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility} | |
.verseblock pre strong{font-weight:400} | |
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex} | |
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic} | |
.quoteblock .attribution br,.verseblock .attribution br{display:none} | |
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.05em;color:rgba(0,0,0,.6)} | |
.quoteblock.abstract{margin:0 0 1.25em 0;display:block} | |
.quoteblock.abstract blockquote,.quoteblock.abstract blockquote p{text-align:left;word-spacing:0} | |
.quoteblock.abstract blockquote:before,.quoteblock.abstract blockquote p:first-of-type:before{display:none} | |
table.tableblock{max-width:100%;border-collapse:separate} | |
table.tableblock td>.paragraph:last-child p>p:last-child,table.tableblock th>p:last-child,table.tableblock td>p:last-child{margin-bottom:0} | |
table.spread{width:100%} | |
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede} | |
table.grid-all th.tableblock,table.grid-all td.tableblock{border-width:0 1px 1px 0} | |
table.grid-all tfoot>tr>th.tableblock,table.grid-all tfoot>tr>td.tableblock{border-width:1px 1px 0 0} | |
table.grid-cols th.tableblock,table.grid-cols td.tableblock{border-width:0 1px 0 0} | |
table.grid-all *>tr>.tableblock:last-child,table.grid-cols *>tr>.tableblock:last-child{border-right-width:0} | |
table.grid-rows th.tableblock,table.grid-rows td.tableblock{border-width:0 0 1px 0} | |
table.grid-all tbody>tr:last-child>th.tableblock,table.grid-all tbody>tr:last-child>td.tableblock,table.grid-all thead:last-child>tr>th.tableblock,table.grid-rows tbody>tr:last-child>th.tableblock,table.grid-rows tbody>tr:last-child>td.tableblock,table.grid-rows thead:last-child>tr>th.tableblock{border-bottom-width:0} | |
table.grid-rows tfoot>tr>th.tableblock,table.grid-rows tfoot>tr>td.tableblock{border-width:1px 0 0 0} | |
table.frame-all{border-width:1px} | |
table.frame-sides{border-width:0 1px} | |
table.frame-topbot{border-width:1px 0} | |
th.halign-left,td.halign-left{text-align:left} | |
th.halign-right,td.halign-right{text-align:right} | |
th.halign-center,td.halign-center{text-align:center} | |
th.valign-top,td.valign-top{vertical-align:top} | |
th.valign-bottom,td.valign-bottom{vertical-align:bottom} | |
th.valign-middle,td.valign-middle{vertical-align:middle} | |
table thead th,table tfoot th{font-weight:bold} | |
tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7} | |
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold} | |
p.tableblock>code:only-child{background:none;padding:0} | |
p.tableblock{font-size:1em} | |
td>div.verse{white-space:pre} | |
ol{margin-left:1.75em} | |
ul li ol{margin-left:1.5em} | |
dl dd{margin-left:1.125em} | |
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0} | |
ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em} | |
ul.unstyled,ol.unnumbered,ul.checklist,ul.none{list-style-type:none} | |
ul.unstyled,ol.unnumbered,ul.checklist{margin-left:.625em} | |
ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1em;font-size:.85em} | |
ul.checklist li>p:first-child>input[type="checkbox"]:first-child{width:1em;position:relative;top:1px} | |
ul.inline{margin:0 auto .625em auto;margin-left:-1.375em;margin-right:0;padding:0;list-style:none;overflow:hidden} | |
ul.inline>li{list-style:none;float:left;margin-left:1.375em;display:block} | |
ul.inline>li>*{display:block} | |
.unstyled dl dt{font-weight:400;font-style:normal} | |
ol.arabic{list-style-type:decimal} | |
ol.decimal{list-style-type:decimal-leading-zero} | |
ol.loweralpha{list-style-type:lower-alpha} | |
ol.upperalpha{list-style-type:upper-alpha} | |
ol.lowerroman{list-style-type:lower-roman} | |
ol.upperroman{list-style-type:upper-roman} | |
ol.lowergreek{list-style-type:lower-greek} | |
.hdlist>table,.colist>table{border:0;background:none} | |
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none} | |
td.hdlist1{padding-right:.75em;font-weight:bold} | |
td.hdlist1,td.hdlist2{vertical-align:top} | |
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em} | |
.colist>table tr>td:first-of-type{padding:0 .75em;line-height:1} | |
.colist>table tr>td:last-of-type{padding:.25em 0} | |
.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd} | |
.imageblock.left,.imageblock[style*="float: left"]{margin:.25em .625em 1.25em 0} | |
.imageblock.right,.imageblock[style*="float: right"]{margin:.25em 0 1.25em .625em} | |
.imageblock>.title{margin-bottom:0} | |
.imageblock.thumb,.imageblock.th{border-width:6px} | |
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em} | |
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0} | |
.image.left{margin-right:.625em} | |
.image.right{margin-left:.625em} | |
a.image{text-decoration:none} | |
span.footnote,span.footnoteref{vertical-align:super;font-size:.875em} | |
span.footnote a,span.footnoteref a{text-decoration:none} | |
span.footnote a:active,span.footnoteref a:active{text-decoration:underline} | |
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em} | |
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em 0;border-width:1px 0 0 0} | |
#footnotes .footnote{padding:0 .375em;line-height:1.3;font-size:.875em;margin-left:1.2em;text-indent:-1.2em;margin-bottom:.2em} | |
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none} | |
#footnotes .footnote:last-of-type{margin-bottom:0} | |
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0} | |
.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0} | |
.gist .file-data>table td.line-data{width:99%} | |
div.unbreakable{page-break-inside:avoid} | |
.big{font-size:larger} | |
.small{font-size:smaller} | |
.underline{text-decoration:underline} | |
.overline{text-decoration:overline} | |
.line-through{text-decoration:line-through} | |
.aqua{color:#00bfbf} | |
.aqua-background{background-color:#00fafa} | |
.black{color:#000} | |
.black-background{background-color:#000} | |
.blue{color:#0000bf} | |
.blue-background{background-color:#0000fa} | |
.fuchsia{color:#bf00bf} | |
.fuchsia-background{background-color:#fa00fa} | |
.gray{color:#606060} | |
.gray-background{background-color:#7d7d7d} | |
.green{color:#006000} | |
.green-background{background-color:#007d00} | |
.lime{color:#00bf00} | |
.lime-background{background-color:#00fa00} | |
.maroon{color:#600000} | |
.maroon-background{background-color:#7d0000} | |
.navy{color:#000060} | |
.navy-background{background-color:#00007d} | |
.olive{color:#606000} | |
.olive-background{background-color:#7d7d00} | |
.purple{color:#600060} | |
.purple-background{background-color:#7d007d} | |
.red{color:#bf0000} | |
.red-background{background-color:#fa0000} | |
.silver{color:#909090} | |
.silver-background{background-color:#bcbcbc} | |
.teal{color:#006060} | |
.teal-background{background-color:#007d7d} | |
.white{color:#bfbfbf} | |
.white-background{background-color:#fafafa} | |
.yellow{color:#bfbf00} | |
.yellow-background{background-color:#fafa00} | |
span.icon>.fa{cursor:default} | |
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default} | |
.admonitionblock td.icon .icon-note:before{content:"\f05a";color:#19407c} | |
.admonitionblock td.icon .icon-tip:before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111} | |
.admonitionblock td.icon .icon-warning:before{content:"\f071";color:#bf6900} | |
.admonitionblock td.icon .icon-caution:before{content:"\f06d";color:#bf3400} | |
.admonitionblock td.icon .icon-important:before{content:"\f06a";color:#bf0000} | |
.conum[data-value]{display:inline-block;color:#fff!important;background-color:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold} | |
.conum[data-value] *{color:#fff!important} | |
.conum[data-value]+b{display:none} | |
.conum[data-value]:after{content:attr(data-value)} | |
pre .conum[data-value]{position:relative;top:-.125em} | |
b.conum *{color:inherit!important} | |
.conum:not([data-value]):empty{display:none} | |
h1,h2{letter-spacing:-.01em} | |
dt,th.tableblock,td.content{text-rendering:optimizeLegibility} | |
p,td.content{letter-spacing:-.01em} | |
p strong,td.content strong{letter-spacing:-.005em} | |
p,blockquote,dt,td.content{font-size:1.0625rem} | |
p{margin-bottom:1.25rem} | |
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em} | |
.exampleblock>.content{background-color:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc} | |
.print-only{display:none!important} | |
@media print{@page{margin:1.25cm .75cm} | |
*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important} | |
a{color:inherit!important;text-decoration:underline!important} | |
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important} | |
a[href^="http:"]:not(.bare):after,a[href^="https:"]:not(.bare):after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em} | |
abbr[title]:after{content:" (" attr(title) ")"} | |
pre,blockquote,tr,img{page-break-inside:avoid} | |
thead{display:table-header-group} | |
img{max-width:100%!important} | |
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3} | |
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid} | |
#toc,.sidebarblock,.exampleblock>.content{background:none!important} | |
#toc{border-bottom:1px solid #ddddd8!important;padding-bottom:0!important} | |
.sect1{padding-bottom:0!important} | |
.sect1+.sect1{border:0!important} | |
#header>h1:first-child{margin-top:1.25rem} | |
body.book #header{text-align:center} | |
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em 0} | |
body.book #header .details{border:0!important;display:block;padding:0!important} | |
body.book #header .details span:first-child{margin-left:0!important} | |
body.book #header .details br{display:block} | |
body.book #header .details br+span:before{content:none!important} | |
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important} | |
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always} | |
.listingblock code[data-lang]:before{display:block} | |
#footer{background:none!important;padding:0 .9375em} | |
#footer-text{color:rgba(0,0,0,.6)!important;font-size:.9em} | |
.hide-on-print{display:none!important} | |
.print-only{display:block!important} | |
.hide-for-print{display:none!important} | |
.show-for-print{display:inherit!important}} | |
div.paragraph.indented p {padding-left: 3em;} | |
div.paragraph.indented2 p {padding-left: 6em;} | |
div.paragraph.indented3 p {padding-left: 9em;} | |
</style> | |
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.5.0/css/font-awesome.min.css"> | |
<style> | |
/* Stylesheet for CodeRay to match GitHub theme | MIT License | http://foundation.zurb.com */ | |
/*pre.CodeRay {background-color:#f7f7f8;}*/ | |
.CodeRay .line-numbers{border-right:1px solid #d8d8d8;padding:0 0.5em 0 .25em} | |
.CodeRay span.line-numbers{display:inline-block;margin-right:.5em;color:rgba(0,0,0,.3)} | |
.CodeRay .line-numbers strong{color:rgba(0,0,0,.4)} | |
table.CodeRay{border-collapse:separate;border-spacing:0;margin-bottom:0;border:0;background:none} | |
table.CodeRay td{vertical-align: top;line-height:1.45} | |
table.CodeRay td.line-numbers{text-align:right} | |
table.CodeRay td.line-numbers>pre{padding:0;color:rgba(0,0,0,.3)} | |
table.CodeRay td.code{padding:0 0 0 .5em} | |
table.CodeRay td.code>pre{padding:0} | |
.CodeRay .debug{color:#fff !important;background:#000080 !important} | |
.CodeRay .annotation{color:#007} | |
.CodeRay .attribute-name{color:#000080} | |
.CodeRay .attribute-value{color:#700} | |
.CodeRay .binary{color:#509} | |
.CodeRay .comment{color:#998;font-style:italic} | |
.CodeRay .char{color:#04d} | |
.CodeRay .char .content{color:#04d} | |
.CodeRay .char .delimiter{color:#039} | |
.CodeRay .class{color:#458;font-weight:bold} | |
.CodeRay .complex{color:#a08} | |
.CodeRay .constant,.CodeRay .predefined-constant{color:#008080} | |
.CodeRay .color{color:#099} | |
.CodeRay .class-variable{color:#369} | |
.CodeRay .decorator{color:#b0b} | |
.CodeRay .definition{color:#099} | |
.CodeRay .delimiter{color:#000} | |
.CodeRay .doc{color:#970} | |
.CodeRay .doctype{color:#34b} | |
.CodeRay .doc-string{color:#d42} | |
.CodeRay .escape{color:#666} | |
.CodeRay .entity{color:#800} | |
.CodeRay .error{color:#808} | |
.CodeRay .exception{color:inherit} | |
.CodeRay .filename{color:#099} | |
.CodeRay .function{color:#900;font-weight:bold} | |
.CodeRay .global-variable{color:#008080} | |
.CodeRay .hex{color:#058} | |
.CodeRay .integer,.CodeRay .float{color:#099} | |
.CodeRay .include{color:#555} | |
.CodeRay .inline{color:#000} | |
.CodeRay .inline .inline{background:#ccc} | |
.CodeRay .inline .inline .inline{background:#bbb} | |
.CodeRay .inline .inline-delimiter{color:#d14} | |
.CodeRay .inline-delimiter{color:#d14} | |
.CodeRay .important{color:#555;font-weight:bold} | |
.CodeRay .interpreted{color:#b2b} | |
.CodeRay .instance-variable{color:#008080} | |
.CodeRay .label{color:#970} | |
.CodeRay .local-variable{color:#963} | |
.CodeRay .octal{color:#40e} | |
.CodeRay .predefined{color:#369} | |
.CodeRay .preprocessor{color:#579} | |
.CodeRay .pseudo-class{color:#555} | |
.CodeRay .directive{font-weight:bold} | |
.CodeRay .type{font-weight:bold} | |
.CodeRay .predefined-type{color:inherit} | |
.CodeRay .reserved,.CodeRay .keyword {color:#000;font-weight:bold} | |
.CodeRay .key{color:#808} | |
.CodeRay .key .delimiter{color:#606} | |
.CodeRay .key .char{color:#80f} | |
.CodeRay .value{color:#088} | |
.CodeRay .regexp .delimiter{color:#808} | |
.CodeRay .regexp .content{color:#808} | |
.CodeRay .regexp .modifier{color:#808} | |
.CodeRay .regexp .char{color:#d14} | |
.CodeRay .regexp .function{color:#404;font-weight:bold} | |
.CodeRay .string{color:#d20} | |
.CodeRay .string .string .string{background:#ffd0d0} | |
.CodeRay .string .content{color:#d14} | |
.CodeRay .string .char{color:#d14} | |
.CodeRay .string .delimiter{color:#d14} | |
.CodeRay .shell{color:#d14} | |
.CodeRay .shell .delimiter{color:#d14} | |
.CodeRay .symbol{color:#990073} | |
.CodeRay .symbol .content{color:#a60} | |
.CodeRay .symbol .delimiter{color:#630} | |
.CodeRay .tag{color:#008080} | |
.CodeRay .tag-special{color:#d70} | |
.CodeRay .variable{color:#036} | |
.CodeRay .insert{background:#afa} | |
.CodeRay .delete{background:#faa} | |
.CodeRay .change{color:#aaf;background:#007} | |
.CodeRay .head{color:#f8f;background:#505} | |
.CodeRay .insert .insert{color:#080} | |
.CodeRay .delete .delete{color:#800} | |
.CodeRay .change .change{color:#66f} | |
.CodeRay .head .head{color:#f4f} | |
</style> | |
</head> | |
<body class="book toc2 toc-left"> | |
<div id="header"> | |
<h1>odb User Guide</h1> | |
<div class="details"> | |
<span id="revnumber">version 2.3.0</span> | |
</div> | |
<div id="toc" class="toc2"> | |
<div id="toctitle">Table of Contents</div> | |
<ul class="sectlevel1"> | |
<li><a href="#_about_this_document">1. About This Document</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_intended_audience">1.1. Intended Audience</a></li> | |
<li><a href="#_new_and_changed_information">1.2. New and Changed Information</a></li> | |
<li><a href="#_notation_conventions">1.3. Notation Conventions</a></li> | |
<li><a href="#_comments_encouraged">1.4. Comments Encouraged</a></li> | |
</ul> | |
</li> | |
<li><a href="#_introduction">2. Introduction</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_what_is_odb">2.1. What is odb</a></li> | |
</ul> | |
</li> | |
<li><a href="#_installation_and_configuration">3. Installation and Configuration</a></li> | |
<li><a href="#_basic_concepts">4. Basic Concepts</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_get_help">4.1. Get Help</a></li> | |
<li><a href="#_connect_to_database">4.2. Connect to Database</a></li> | |
<li><a href="#_list_odbc_drivers_and_data_sources">4.3. List ODBC Drivers and Data Sources</a></li> | |
<li><a href="#_obtain_database_information">4.4. Obtain Database Information</a></li> | |
<li><a href="#_list_database_objects">4.5. List Database Objects</a></li> | |
<li><a href="#_perform_actions_on_multiple_database_objects">4.6. Perform Actions on Multiple Database Objects</a></li> | |
<li><a href="#_run_commands_and_scripts">4.7. Run Commands and Scripts</a></li> | |
<li><a href="#_shell_script_here_document_syntax">4.8. Shell Script "here document" Syntax</a></li> | |
<li><a href="#_parallelize_multiple_commands_and_scripts">4.9. Parallelize Multiple Commands and Scripts</a></li> | |
<li><a href="#_limit_number_of_odb_threads">4.10. Limit Number of odb Threads</a></li> | |
<li><a href="#_change_executions_distributed_across_threads">4.11. Change Executions Distributed Across Threads</a></li> | |
<li><a href="#concepts_load_balancing">4.12. Dynamic Load Balancing</a></li> | |
<li><a href="#_use_variables_in_odb_scripts">4.13. Use Variables in odb Scripts</a></li> | |
<li><a href="#_thread_id_thread_execution_and_script_command">4.14. Thread ID, Thread Execution#, and Script Command#</a></li> | |
<li><a href="#_validate_sql_scripts">4.15. Validate SQL Scripts</a></li> | |
<li><a href="#_different_data_sources_for_different_threads">4.16. Different Data Sources for Different Threads</a></li> | |
<li><a href="#_format_query_results">4.17. Format Query Results</a></li> | |
<li><a href="#_extract_table_ddl">4.18. Extract Table DDL</a></li> | |
</ul> | |
</li> | |
<li><a href="#_load_extract_copy">5. Load, Extract, Copy</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_load_files">5.1. Load Files</a> | |
<ul class="sectlevel3"> | |
<li><a href="#_data_loading_operators">5.1.1. Data Loading Operators</a></li> | |
</ul> | |
</li> | |
<li><a href="#load_map_fields">5.2. Map Source File Fields to Target Table Columns</a></li> | |
<li><a href="#load_mapfiles_ignore">5.3. Use mapfiles to Ignore and/or Transform Fields When Loading</a></li> | |
<li><a href="#_use_mapfiles_to_load_fixed_format_files">5.4. Use mapfiles to Load Fixed Format Files</a></li> | |
<li><a href="#_generate_and_load_data">5.5. Generate and Load Data</a></li> | |
<li><a href="#load_default_values">5.6. Load Default Values</a></li> | |
<li><a href="#load_binary_files">5.7. Load Binary Files</a></li> | |
<li><a href="#load_xml_files">5.8. Load XML Files</a> | |
<ul class="sectlevel3"> | |
<li><a href="#_load_xml_files_where_data_is_stored_in_element_nodes">5.8.1. Load XML Files Where Data is Stored in Element Nodes</a></li> | |
<li><a href="#_load_xml_files_where_data_is_stored_in_attribute_nodes">5.8.2. Load XML Files Where Data is Stored in Attribute Nodes</a></li> | |
</ul> | |
</li> | |
<li><a href="#_reduce_the_odbc_buffer_size">5.9. Reduce the ODBC Buffer Size</a></li> | |
<li><a href="#_extract_tables">5.10. Extract Tables</a> | |
<ul class="sectlevel3"> | |
<li><a href="#_extraction_options">5.10.1. Extraction Options</a></li> | |
</ul> | |
</li> | |
<li><a href="#_extract_a_list_of_tables">5.11. Extract a List of Tables</a></li> | |
<li><a href="#_copy_tables_from_one_database_to_another">5.12. Copy Tables From One Database to Another</a> | |
<ul class="sectlevel3"> | |
<li><a href="#_copy_operators">5.12.1. Copy Operators</a></li> | |
</ul> | |
</li> | |
<li><a href="#_copy_a_list_of_tables">5.13. Copy a List of Tables</a></li> | |
<li><a href="#_case_sensitive_table_and_column_names">5.14. Case-Sensitive Table and Column Names</a></li> | |
<li><a href="#_determine_appropriate_number_of_threads_for_load_extract_copy_diff">5.15. Determine Appropriate Number of Threads for Load/Extract/Copy/Diff</a></li> | |
<li><a href="#_integrating_with_hadoop">5.16. Integrating With Hadoop</a></li> | |
</ul> | |
</li> | |
<li><a href="#_comparing_tables_technology_preview">6. Comparing Tables (Technology Preview)</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_diff_operators">6.1. Diff Operators</a></li> | |
</ul> | |
</li> | |
<li><a href="#_odb_as_a_query_driver_technology_preview">7. odb as a Query Driver (Technology Preview)</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_getting_csv_output">7.1. Getting CSV Output</a></li> | |
<li><a href="#_assign_label_to_a_query">7.2. Assign Label to a Query</a></li> | |
<li><a href="#query_driver_all_scripts_path">7.3. Run All Scripts With a Given Path</a></li> | |
<li><a href="#_randomizing_execution_order">7.4. Randomizing Execution Order</a></li> | |
<li><a href="#_defining_a_timeout">7.5. Defining a Timeout</a></li> | |
<li><a href="#_simulating_user_thinking_time">7.6. Simulating User Thinking Time</a></li> | |
<li><a href="#_starting_threads_gracefully">7.7. Starting Threads Gracefully</a></li> | |
<li><a href="#_re_looping_a_given_workload">7.8. Re-looping a Given Workload</a></li> | |
</ul> | |
</li> | |
<li><a href="#_odb_as_a_sql_interpreter_technology_preview">8. odb as a SQL Interpreter (Technology Preview)</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_main_odb_sql_interpreter_features">8.1. Main odb SQL Interpreter Features</a> | |
<ul class="sectlevel3"> | |
<li><a href="#_odb_sql_interpreter_help">8.1.1. odb SQL Interpreter help</a></li> | |
</ul> | |
</li> | |
<li><a href="#sql_run_commands">8.2. Run Commands When the Interpreter Starts</a></li> | |
<li><a href="#sql_custom_prompt">8.3. Customizing the Interpreter Prompt</a></li> | |
</ul> | |
</li> | |
<li><a href="#_appendixes">9. Appendixes</a> | |
<ul class="sectlevel2"> | |
<li><a href="#_a_troubleshooting">9.1. A. Troubleshooting</a></li> | |
<li><a href="#_b_develop_and_test_odb">9.2. B. Develop and Test odb</a> | |
<ul class="sectlevel3"> | |
<li><a href="#_develop">9.2.1. Develop</a></li> | |
<li><a href="#_test">9.2.2. Test</a></li> | |
</ul> | |
</li> | |
</ul> | |
</li> | |
</ul> | |
</div> | |
</div> | |
<div id="content"> | |
<div id="preamble"> | |
<div class="sectionbody"> | |
<div class="paragraph"> | |
<p><strong>License Statement</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file | |
distributed with this work for additional information regarding copyright ownership. The ASF licenses this file | |
to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with | |
the License. You may obtain a copy of the License at <a href="http://www.apache.org/licenses/LICENSE-2.0" class="bare">http://www.apache.org/licenses/LICENSE-2.0</a></p> | |
</div> | |
<div class="paragraph"> | |
<p>Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an | |
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the | |
specific language governing permissions and limitations under the License.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Revision History</strong></p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 50%;"> | |
<col style="width: 50%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Version</th> | |
<th class="tableblock halign-left valign-top">Date</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">2.2.0</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">TBD</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">2.1.0</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">May 1, 2017</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">2.0.1</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">July 7, 2016</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">2.0.0</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">June 6, 2016</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">1.3.0</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">January, 2016</p></td> | |
</tr> | |
</tbody> | |
</table> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_about_this_document">1. About This Document</h2> | |
<div class="sectionbody"> | |
<div class="paragraph"> | |
<p>This guide describes how to use odb, a multi-threaded, ODBC-based command-line tool, to perform various operations on a Trafodion | |
database.</p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
In the current release of Trafodion, only loading, extracting, and copying data operations are production ready, meaning that that have | |
been fully tested and are ready to be used in a production environment.<br> | |
<br> | |
Other features are designated as <em>Technology Preview</em> meaning that they have not been fully tested and are not ready for production use. | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div class="sect2"> | |
<h3 id="_intended_audience">1.1. Intended Audience</h3> | |
<div class="paragraph"> | |
<p>This guide is intended for database administrators and other users who want to run scripts that operate on a Trafodion database, primarily for | |
parallel data loading.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_new_and_changed_information">1.2. New and Changed Information</h3> | |
<div class="paragraph"> | |
<p>This manual guide is new.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_notation_conventions">1.3. Notation Conventions</h3> | |
<div class="paragraph"> | |
<p>This list summarizes the notation conventions for syntax presentation in this manual.</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>UPPERCASE LETTERS</p> | |
<div class="paragraph"> | |
<p>Uppercase letters indicate keywords and reserved words. Type these items exactly as shown. Items not enclosed in brackets are required.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">SELECT</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>lowercase letters</p> | |
<div class="paragraph"> | |
<p>Lowercase letters, regardless of font, indicate variable items that you supply. Items not enclosed in brackets are required.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">file-name</code></pre> | |
</div> | |
</div> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>[ ] Brackets</p> | |
<div class="paragraph"> | |
<p>Brackets enclose optional syntax items.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">DATETIME [start-field TO] end-field</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>A group of items enclosed in brackets is a list from which you can choose one item or none.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The items in the list can be arranged either vertically, with aligned brackets on each side of the list, or horizontally, enclosed in a pair of brackets and separated by vertical lines.</p> | |
</div> | |
<div class="paragraph"> | |
<p>For example:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">DROP SCHEMA schema [CASCADE] | |
DROP SCHEMA schema [ CASCADE | RESTRICT ]</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>{ } Braces</p> | |
<div class="paragraph"> | |
<p>Braces enclose required syntax items.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">FROM { grantee [, grantee ] ... }</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>A group of items enclosed in braces is a list from which you are required to choose one item.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The items in the list can be arranged either vertically, with aligned braces on each side of the list, or horizontally, enclosed in a pair of braces and separated by vertical lines.</p> | |
</div> | |
<div class="paragraph"> | |
<p>For example:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">INTERVAL { start-field TO end-field } | |
{ single-field } | |
INTERVAL { start-field TO end-field | single-field }</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>| Vertical Line</p> | |
<div class="paragraph"> | |
<p>A vertical line separates alternatives in a horizontal list that is enclosed in brackets or braces.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">{expression | NULL}</code></pre> | |
</div> | |
</div> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>… Ellipsis</p> | |
<div class="paragraph"> | |
<p>An ellipsis immediately following a pair of brackets or braces indicates that you can repeat the enclosed sequence of syntax items any number of times.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">ATTRIBUTE[S] attribute [, attribute] ... | |
{, sql-expression } ...</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>An ellipsis immediately following a single syntax item indicates that you can repeat that syntax item any number of times.</p> | |
</div> | |
<div class="paragraph"> | |
<p>For example:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">expression-n ...</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>Punctuation</p> | |
<div class="paragraph"> | |
<p>Parentheses, commas, semicolons, and other symbols not previously described must be typed as shown.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">DAY (datetime-expression) | |
@script-file</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Quotation marks around a symbol such as a bracket or brace indicate the symbol is a required character that you must type as shown.</p> | |
</div> | |
<div class="paragraph"> | |
<p>For example:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">"{" module-name [, module-name] ... "}"</code></pre> | |
</div> | |
</div> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Item Spacing</p> | |
<div class="paragraph"> | |
<p>Spaces shown between items are required unless one of the items is a punctuation symbol such as a parenthesis or a comma.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">DAY (datetime-expression) DAY(datetime-expression)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>If there is no space between two items, spaces are not permitted. In this example, no spaces are permitted between the period and any other items:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">myfile.sh</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>Line Spacing</p> | |
<div class="paragraph"> | |
<p>If the syntax of a command is too long to fit on a single line, each continuation line is indented three spaces and is separated from the preceding line by a blank line.</p> | |
</div> | |
<div class="paragraph"> | |
<p>This spacing distinguishes items in a continuation line from items in a vertical list of selections.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">match-value [NOT] LIKE _pattern | |
[ESCAPE esc-char-expression]</code></pre> | |
</div> | |
</div> | |
</li> | |
</ul> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_comments_encouraged">1.4. Comments Encouraged</h3> | |
<div class="paragraph"> | |
<p>We encourage your comments concerning this document. We are committed to providing documentation that meets your | |
needs. Send any errors found, suggestions for improvement, or compliments to <a href="mailto:user@trafodion.apache.org">user@trafodion.apache.org</a>.</p> | |
</div> | |
<div class="paragraph"> | |
<p>Include the document title and any comment, error found, or suggestion for improvement you have concerning this document.</p> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_introduction">2. Introduction</h2> | |
<div class="sectionbody"> | |
<div class="sect2"> | |
<h3 id="_what_is_odb">2.1. What is odb</h3> | |
<div class="paragraph"> | |
<p>odb is a platform independent, multi-threaded, ODBC command-line tool you can use as a:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Parallel data loader/extractor</p> | |
</li> | |
<li> | |
<p>Query driver (Technology Preview)</p> | |
</li> | |
<li> | |
<p>SQL interpreter (Technology Preview)</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>odb is written in ANSI C. Currently, odb is available only in a 64-bit version for the Linux platform, | |
linked to the unixODBC driver manager.</p> | |
</div> | |
<div class="paragraph"> | |
<p>odb executables use the following naming convention, <code><strong><em>odbAABCC</em></strong></code>, where:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code><strong>AA</strong></code> can be 64 (bit) (32 bit is not currently supported).</p> | |
</li> | |
<li> | |
<p><code><strong>B</strong></code> identifies the platform/compiler:</p> | |
</li> | |
<li> | |
<p><code>l</code> = Linux/gcc</p> | |
</li> | |
<li> | |
<p><code>w</code> = Windows/MS Visual Studio (not yet tested)</p> | |
</li> | |
<li> | |
<p><code><strong>CC</strong></code> identifies the ODBC Driver Manager to which odb was linked:</p> | |
</li> | |
<li> | |
<p><code>uo</code> = unixODBC Driver Manager</p> | |
</li> | |
<li> | |
<p><code>ms</code> = Microsoft ODBC Driver Manager (not yet tested)</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>So, for example:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code><strong>odb64luo</strong></code> is the 64-bit executable for Linux linked with the unixODBC Driver Manager.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>This document contains examples run with the <code><strong>odb64luo</strong></code> executable.</p> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_installation_and_configuration">3. Installation and Configuration</h2> | |
<div class="sectionbody"> | |
<div class="paragraph"> | |
<p>See the <a href="http://trafodion.apache.org/docs/odb/index.html">Trafodion Client Installation Guide</a> for install instructions.</p> | |
</div> | |
<div class="paragraph"> | |
<p>Refer to the <a href="http://www.unixodbc.org/doc/">unixODBC documentation</a> for additional | |
information for <code>unixODBC</code>.</p> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_basic_concepts">4. Basic Concepts</h2> | |
<div class="sectionbody"> | |
<div class="sect2"> | |
<h3 id="_get_help">4.1. Get Help</h3> | |
<div class="paragraph"> | |
<p>The following command shows the odb help:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -h | |
odb version 1.3.0 | |
Build: linux, amd64, gcc generic m64, uodbc, mreadline, dynamic gzip, dynamic libhdfs, dynamic libxml2 [Mar 30 2015 00:29:25] | |
-h: print this help | |
-version: print odb version and exit | |
-lsdrv: list available drivers @ Driver Manager level | |
-lsdsn: list available Data Sources | |
Connection related options. You can connect using either: | |
-u User: (default $ODB_USER variable) | |
-p Password: (default $ODB_PWD variable) | |
-d Data_Source_Name: (default $ODB_DSN variable) | |
-ca Connection_Attributes (normally used instead of -d DSN) | |
-U sets SQL_TXN_READ_UNCOMMITTED isolation level | |
-ndsn [+]<number>: adds 1 to <number> to DSN | |
-nps <nbytes>[:<nbytes>]: specify source[:target] network packet size | |
SQL interpreter options: | |
-I [$ODB_INI SECTION]: interactive mode shell | |
-noconnect: do not connect on startup General options: | |
-q [cmd|res|all|off]: do not print commands/results/both | |
-i [TYPE[MULT,WIDE_MULT]:CATALOG.SCHEMA[.TABLE]]: lists following object types: | |
(t)ables, (v)iews, s(y)nonyns, (s)chemas, (c)atalogs, syst(e)m tables | |
(l)ocal temp, (g)lobal temp, (m)at views, (M)mat view groups, (a)lias | |
(A)ll object types, (T)table desc, (D)table DDL, (U) table DDL with multipliers | |
-r #rowset: rowset to be used insert/selects (default 100) | |
-soe: Stop On Error (script execution/loading task) | |
-N : Null run. Doesn't SQLExecute statements | |
-v : be verbose | |
-vv : Print execution table | |
-noschema : do not use schemas: CAT.OBJ instead of CAT.SCH.OBJ | |
-nocatalog : do not use catalogs: SCH.OBJ instead of CAT.SCH.OBJ | |
-nocatnull : like -nocatalog but uses NULL instead of empty CAT strings | |
-ucs2toutf8 : set UCS-2 to UTF-8 conversion in odb | |
-var var_name var_value: set user defined variables | |
-ksep char/code: Thousands Separator Character (default ',') | |
-dsep char/code: Decimal Separator Character (default '.') | |
SQL execution options [connection required]: | |
-x [#inst:]'command': runs #inst (default 1) command instances | |
-f [#inst:]'script': runs #inst (default 1) script instances | |
-P script_path_regexp: runs in parallel scripts_path_regexp if script_path_regexp ends with / all files in that dir | |
-S script_path_regexp: runs serially scripts_path_regexp if script_path_regexp ends with / all files in that dir | |
-L #loops: runs everything #loops times | |
-T max_threads: max number of execution threads | |
-dlb: use Dynamic Load Balancing | |
-timeout #seconds: stops everything after #seconds (no Win32) | |
-delay #ms: delay (ms) before starting next thread | |
-ldelay #ms: delay (ms) before starting next loop in a thread | |
-ttime #ms[:ms]: delay (ms) before starting next command in a thread random delay if a [min:max] range is specified | |
-F #records: max rows to fetch | |
-c : output in csv format | |
-b : print start time in the headers when CSV output | |
-pcn: Print Column Names | |
-plm: Print Line Mode | |
-fs char/code: Field Sep <char> ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
-rs char/code: Rec Sep <char> ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
-sq char/code: String Qualifier (default none) | |
-ec char/code: Escape Character (default '\') | |
-ns nullstring: print nullstring when a field is NULL | |
-trim: Trim leading/trailing white spaces from txt cols | |
-drs: describe result set (#cols, data types...) for each Q) | |
-hint: do not remove C style comments (treat them as hints) | |
-casesens: set case sensitive DB | |
-Z : shuffle the execution table randomizing Qs start order | |
Data loading options [connection required]: | |
-l src=[-]file:tgt=table[:map=mapfile][:fs=fieldsep][:rs=recsep][:soe] | |
[:skip=linestoskip][:ns=nullstring][:ec=eschar][:sq=stringqualifier] | |
[:pc=padchar][:em=embedchar][:errmax=#max_err][:commit=auto|end|#rows|x#rs] | |
[:rows=#rowset][:norb][:full][:max=#max_rec][:truncate][:show][:bpc=#][:bpwc=#] | |
[:nomark][:parallel=number][:iobuff=#size][:buffsz=#size]][:fieldtrunc={0-4}] | |
[:pre={@sqlfile}|{[sqlcmd]}][:post={@sqlfile}|{[sqlcmd]}][:ifempty] | |
[:direct][:bad=[+]badfile][:tpar=#tables][:maxlen=#bytes][:time][:loadcmd=IN|UP|UL] | |
[:xmltag=[+]element][:xmlord][:xmldump] | |
Defaults/notes: | |
* src file: local file or {hdfs,mapr}[@host,port[,huser]].<HDFS_PATH> | |
* fs: default ','. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* rs: default '\n'. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* ec: default '\'. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* pc: no default. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* direct: only for Vertica databases | |
* bpc: default 1,bpwc: default 4 | |
* loadcmd: default IN. only for {project-name} databases | |
Data extraction options [connection required]: | |
-e {src={table|-file}|sql=<custom sql>}:tgt=[+]file[:pwhere=where_cond] | |
[:fs=fieldsep][:rs=recsep][:sq=stringqualifier][:ec=escape_char][:soe] | |
[:ns=nullstring][es=emptystring][:rows=#rowset][:nomark][:binary][:bpc=#][:bpwc=#] | |
[:max=#max_rec][:[r]trim[+]][:cast][:multi][parallel=number][:gzip[=lev]] | |
[:splitby=column][:uncommitted][:iobuff=#size][hblock=#size][:ucs2toutf8] | |
[:pre={@sqlfile}|{[sqlcmd]}[:mpre={@sqlfile}|{[sqlcmd]}[:post={@sqlfile}|{[sqlcmd]}] | |
[tpar=#tables][:time][:cols=[-]columns]][:maxlen=#bytes][:xml] | |
Defaults/notes: | |
* tgt file: local file or {hdfs,mapr}.[@host,port[,huser]].<HDFS_PATH> | |
* fs: default ','. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* rs: default '\n'. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* ec: default '\'. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* sq: no default. Also <ASCII_dec> 0<ASCII_OCT> X<ASCII_HEX> | |
* gzip compression level between 0 and 9 | |
* bpc: default 1,bpwc: default 4 | |
Data copy options [connection required]: | |
-cp src={table|-file:tgt=schema[.table][pwhere=where_cond][:soe][:roe=#][:roedel=#ms] | |
[:truncate][:rows=#rowset][:nomark][:max=#max_rec][:bpc=#][:bpwc=#][:[r]trim[+]] | |
[:parallel=number][errmax=#max_err][:commit=auto|end|#rows|x#rs][:time] ][:cast] | |
[:direct][:uncommitted][:norb][:splitby=column][:pre={@sqlfile}|{[sqlcmd]}] | |
[:post={@sqlfile}|{[sqlcmd]}][:mpre={@sqlfile}|{[sqlcmd]}][:ifempty] | |
[:loaders=#loaders][:tpar=#tables][:cols=[-]columns][:errdmp=file] ][:loadcmd=IN|UP|UL] | |
[sql={[sqlcmd]|@sqlfile|-file}[:bind=auto|char|cdef][:seq=field#[,start]] | |
[tmpre={@sqlfile}|{[sqlcmd]}][:ucs2toutf8=[skip,force,cpucs2,qmark]] | |
Defaults/notes: | |
* loaders: default 2 load threads for each 'extractor' | |
* direct: only work if target database is Vertica | |
* ucs2toutf8: default is 'skip' | |
* roe: default 3 if no arguments | |
* bpc: default 1,bpwc: default 4 | |
* loadcmd: default IN. only for {project-name} databases | |
Data pipe options [connection required]: | |
-pipe sql={[sqlcmd]|@sqlscript|-file}:tgtsql={@sqlfile|[sqlcmd]}[:soe] | |
[:rows=#rowset][:nomark][:max=#max_rec][:bpc=#][:bpwc=#][:errdmp=file] | |
[:parallel=number][errmax=#max_err][:commit=auto|end|#rows|x#rs][:time] | |
[:pre={@sqlfile}|{[sqlcmd]}][:post={@sqlfile}|{[sqlcmd]}] | |
[:mpre={@sqlfile}|{[sqlcmd]}][:tmpre={@sqlfile}|{[sqlcmd]}] | |
[:loaders=#loaders][:tpar=#tables][:bind=auto|char|cdef] | |
Defaults/notes: | |
* loaders: default 1 load threads for each extraction thread | |
* bpc: default 1,bpwc: default 4</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">Table diff options [connection required]: | |
-diff src={table|-file}:tgt=table:[key=columns][:output=[+]file][:pwhere=where_cond] | |
[:pwhere=where_cond][:nomark][:rows=#rowset][:odad][:fs=fieldsep][:time][trim[+]] | |
[:rs=recsep][:quick][:splitby=column][:parallel=number][:max=#max_rec] | |
[:print=[I][D][C]][:ns=nullstring][:es=emptystring][:bpc=#][:bpwc=#][:uncommitted] | |
[:pre={@sqlfile}|{[sqlcmd]}][:post={@sqlfile}|{[sqlcmd]}][tpar=#tables] | |
Defaults/notes: | |
* bpc: default 1,bpwc: default 4 | |
* print: default is Inserted Deleted Changed</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_connect_to_database">4.2. Connect to Database</h3> | |
<div class="paragraph"> | |
<p>odb uses standard ODBC APIs to connect to a database.</p> | |
</div> | |
<div class="paragraph"> | |
<p>Normally you have to provide the following information: user, password and ODBC data source.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo –u user –p password –d dsn ...</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You can provide Driver-specific connection attributes using th <code>-ca</code> command line option.</p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
Command-line passwords are protected against <code>ps -ef</code> sniffing attacks under *nix. You can safely pass your | |
password via <code>–p</code>. An alternative approach is to use environment variables or the odb password prompt (see below). | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div class="paragraph"> | |
<p>odb will use the following environment variables (if defined):</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 15.7894%;"> | |
<col style="width: 52.6315%;"> | |
<col style="width: 31.5791%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Variable</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
<th class="tableblock halign-left valign-top">Corresponding Command-Line Option</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ODB_USER</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">User name to use for database connections</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-u <user></code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ODB_PWD</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for database connections</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-p <passwd></code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ODB_DSN</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">DSN for database connection</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-d <dsn></code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ODB_INI</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Init file for interactive shell</p></td> | |
<td class="tableblock halign-left valign-top"></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ODB_HIST</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">history file name to save command history on exit</p></td> | |
<td class="tableblock halign-left valign-top"></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
Command-line options take precedence over environment variables. | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_list_odbc_drivers_and_data_sources">4.3. List ODBC Drivers and Data Sources</h3> | |
<div class="paragraph"> | |
<p>You can list available drivers with <code>-lsdrv</code>:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -lsdrv | |
Trafodion - Description=Trafodion ODBC Stand Alone Driver | |
...</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You can list locally configured data sources with <code>-lsdsn</code>:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -lsdsn | |
traf - Trafodion | |
VMFELICI – Vertica | |
...</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_obtain_database_information">4.4. Obtain Database Information</h3> | |
<div class="paragraph"> | |
<p>The <code>-i</code> option allows you to get information about the database you’re connecting to as well as the ODBC driver. | |
It’s a simple way to check your credentials and database connection.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/mauro/odb $ ./odb64luo -u xxx -p xxx -d traf -i | |
odb [2015-04-20 21:20:47]: starting ODBC connection(s)... 0 | |
[odb version 1.3.0] | |
Build: linux, amd64, gcc generic m64, uodbc, mreadline, dynamic gzip, dynamic libhdfs, dynamic libxml2 [Apr 8 2015 16:47:49] | |
DBMS product name (SQL_DBMS_NAME) : Trafodion | |
DBMS product version (SQL_DBMS_VER) : 01.03.0000 | |
Database name (SQL_DATABASE_NAME) : TRAFODION | |
Server name (SQL_SERVER_NAME) : --name-- | |
Data source name (SQL_DATA_SOURCE_NAME) : traf | |
Data source RO (SQL_DATA_SOURCE_READ_ONLY) : N | |
ODBC Driver name (SQL_DRIVER_NAME) : libhpodbc64.so | |
ODBC Driver version (SQL_DRIVER_VER) : 03.00.0000 | |
ODBC Driver level (SQL_DRIVER_ODBC_VER) : 03.51 | |
ODBC Driver Manager version (SQL_DM_VER) : 03.52.0002.0002 | |
ODBC Driver Manager level (SQL_ODBC_VER) : 03.52 | |
Connection Packet Size (SQL_ATTR_PACKET_SIZE): 0 | |
odb [2015-04-20 21:20:48]: exiting. Session Elapsed time 0.229 seconds (00:00:00.229)</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_list_database_objects">4.5. List Database Objects</h3> | |
<div class="paragraph"> | |
<p>The previous section used the <code>-i</code> option without any argument.</p> | |
</div> | |
<div class="paragraph"> | |
<p>This option accepts arguments with the following syntax:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">[TYPE:][CATALOG.SCHEMA][.OBJECT]</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>where type can be:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 40%;"> | |
<col style="width: 60%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Type</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code><missing></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">All database object types</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>A:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">All database object types</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>t:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Tables</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>v:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Views</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>a:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Aliases</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>y:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Synonyms</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>l:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Local Temporary</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>g:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Global Temporary</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>m:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Materialized views</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>M:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Materialized view groups</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>s:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Schemas</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>c:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Catalogs</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>T:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Table descriptions</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>D:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Table DDL</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>U[x,y]:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Table DDL multiplying wide columns by Y and non-wide columns by X</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div style="page-break-after: always;"></div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 40%;"> | |
<col style="width: 60%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Example</th> | |
<th class="tableblock halign-left valign-top">Action</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i c:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all catalogs.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i s:</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all schemas.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i TRAFODION.MFTEST</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all objects in <code>TRAFODION.MFTEST</code> schema.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i t:TRAFODION.MFTEST</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all tables in <code>TRAFODION.MFTEST</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i t:TRAFODION.MFTEST.A%</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all tables in <code>TRAFODION.MFTEST</code> schema staring with <code>A</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i v:TRAFODION.MFTEST</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all views in <code>TRAFODION.MFTEST</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i v:TRAFODION.MFTEST.%_V</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">List all views in <code>TRAFODION.MFTEST</code> ending with <code>_V</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-i T:TRAFODION.MFTEST.STG%</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Describe all tables starting with <code>STG</code> in <code>TRAFODION.MFTEST</code>.</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p><strong>Extended Examples</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/mauro/odb $ ./odb64luo -u MFELICI -p xxx -d MFELICI -i T:TRAFODION.MAURIZIO.T% | |
odb [2011-12-07 14:43:51]: starting (1) ODBC connection(s)... 1 | |
Describing: TRAFODION.MAURIZIO.T1 | |
+------+--------------+----+-------+-------+ | |
|COLUMN|TYPE |NULL|DEFAULT|INDEX | | |
+------+--------------+----+-------+-------+ | |
|ID |INTEGER SIGNED|YES | | | | |
|NAME |CHAR(10) |YES | | | | |
|LASTN |VARCHAR(20) |YES | | | | |
+------+--------------+----+-------+-------+ | |
Describing: TRAFODION.MAURIZIO.T11 | |
+------+--------------+----+-------+-------+ | |
|COLUMN|TYPE |NULL|DEFAULT|INDEX | | |
+------+--------------+----+-------+-------+ | |
|ID |INTEGER SIGNED|NO | |T11 1 U| | |
|NAME |CHAR(10) |YES | | | | |
+------+--------------+----+-------+-------+</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The <code>INDEX</code> column (when using type <code>T</code>) contains the following information:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>name</code> of the <code>INDEX</code> (in Trafodion indexes having the same name as the table are Primary Keys).</p> | |
</li> | |
<li> | |
<p><code>ordinal number</code> to identify the order of that field in the index.</p> | |
</li> | |
<li> | |
<p><code>(U)nique o (M)ultiple</code> values allowed.</p> | |
</li> | |
<li> | |
<p><code>(+)</code> means that more than one index includes that field.</p> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_perform_actions_on_multiple_database_objects">4.6. Perform Actions on Multiple Database Objects</h3> | |
<div class="paragraph"> | |
<p>odb uses extended SQL syntax to execute actions on multiple objects: <code>&<type>:<path></code> - where <code><type></code> is one | |
of the object types listed in the previous section.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 60%;"> | |
<col style="width: 40%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Example</th> | |
<th class="tableblock halign-left valign-top">Action</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>delete from &t:MF%</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Purge ALL tables (t:) staring with <code>M”</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>drop view &v:mftest.%vw</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Drop ALL views (v:) ending with <code>_VW</code> in the schema <code>MFTEST</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>UPDATE STATISTICS FOR TABLE &t:TRAFODION.MFTEST.%</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Update Stats for ALL tables in <code>TRAFODION.MFTEST</code>.</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>You can use this <em>extended</em> SQL syntax in the SQL Interpreter or generic SQL scripts.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_run_commands_and_scripts">4.7. Run Commands and Scripts</h3> | |
<div class="paragraph"> | |
<p>The <code>–x</code> switch can be used to run generic SQL commands. You can also use <code>–f</code> to run SQL scripts:</p> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p><code>-x "SQL command"</code> to run a specific SQL command.</p> | |
</li> | |
<li> | |
<p><code>-f <script></code> to run a script file.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -x "select count(*) from customer" | |
150000 | |
[0.0.0]--- 1 row(s) selected in 0.137s (prep 0.000s, exec 0.137s, 1st fetch 0.000s, | |
fetch 0.000s)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The meaning of <code>[0.0.0]</code> will be explained later.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat script.sql | |
SELECT COUNT(*) FROM T1; | |
-- This is a comment | |
SELECT | |
L_RETURNFLAG | |
, L_LINESTATUS | |
, SUM(L_QUANTITY) AS SUM_QTY | |
, SUM(L_EXTENDEDPRICE) AS SUM_BASE_PRICE | |
, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS SUM_DISC_PRICE | |
, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX)) AS SUM_CHARGE | |
, AVG(L_QUANTITY) AS AVG_QTY | |
, AVG(L_EXTENDEDPRICE) AS AVG_PRICE | |
, AVG(L_DISCOUNT) AS AVG_DISC | |
, COUNT(*) AS COUNT_ORDER | |
FROM | |
LINEITEM | |
WHERE | |
L_SHIPDATE <= DATE '1998-12-01' - INTERVAL '90' DAY | |
GROUP BY | |
L_RETURNFLAG, L_LINESTATUS | |
ORDER BY | |
L_RETURNFLAG, L_LINESTATUS | |
;</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -f script.sql | |
[0.0.0]Executing: 'SELECT COUNT(*) FROM T1;' | |
5 | |
[0.0.0]--- 1 row(s) selected in 0.015s (prep 0.000s, exec 0.015s, 1st fetch -0.000s, | |
fetch -0.000s) | |
[0.0.1]Executing: 'SELECT L_RETURNFLAG, L_LINESTATUS, SUM(L_QUANTITY) AS SUM_QTY, | |
SUM(L_EXTENDEDPRICE) AS SUM_BASE_PRICE, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS | |
SUM_DISC_PRICE, SUM(L_EXTENDEDPRICE*(1- L_DISCOUNT)*(1+L_TAX)) AS SUM_CHARGE, | |
AVG(L_QUANTITY) AS AVG_QTY, AVG(L_EXTENDEDPRICE) AS AVG_PRICE, AVG(L_DISCOUNT) AS | |
AVG_DISC, COUNT(*) AS COUNT_ORDER FROM LINEITEM WHERE L_SHIPDATE <= DATE '1998-12-01' | |
- INTERVAL '90' DAY GROUP BY L_RETURNFLAG, L_LINESTATUS ORDER BY L_RETURNFLAG, | |
L_LINESTATUS;' | |
A,F,37734107.00,56586554400.73,53758257134.8700,55909065222.827692,25.522006, | |
38273.129735,0.049985,1478493 | |
... | |
R,F,37719753.00,56568041380.90,53741292684.6040,55889619119.831932,25.505794, | |
38250.854626,0.050009,1478870 | |
[0.0.1]--- 4 row(s) selected in 21.344s (prep 0.000s, exec 21.344s, 1st fetch 0.000s, fetch 0.000s)</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>You can use the <code>-q</code> switch to omit selected output components.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p><code>–q</code> cmd will not print the <strong><em>commands</em></strong> being executed:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -f script.sql –q cmd | |
5 | |
[0.0.0]--- 1 row(s) selected in 0.015s (prep 0.000s, exec 0.015s, 1st fetch -0.000s, | |
fetch -0.000s) | |
A,F,37734107.00,56586554400.73,53758257134.8700,55909065222.827692,25.522006, | |
38273.129735,0.049985,1478493 | |
... | |
R,F,37719753.00,56568041380.90,53741292684.6040,55889619119.831932,25.505794, | |
38250.854626,0.050009,1478870 | |
[0.0.1]--- 4 row(s) selected in 21.344s (prep 0.000s, exec 21.344s, 1st fetch 0.000s, | |
fetch 0.000s)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>While <code>-q</code> res will nit print the <strong><em>results</em></strong>:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -f script.sql –q res | |
[0.0.0]Executing: 'SELECT COUNT(*) FROM T1;' | |
[0.0.0]--- 1 row(s) selected in 0.015s (prep 0.000s, exec 0.015s, 1st fetch -0.000s, | |
fetch -0.000s) | |
[0.0.1]Executing: 'SELECT L_RETURNFLAG,L_LINESTATUS, SUM(L_QUANTITY) AS | |
SUM_QTY, SUM(L_EXTENDEDPRICE) AS SUM_BASE_PRICE, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) | |
AS SUM_DISC_PRICE, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX)) AS SUM_CHARGE, | |
AVG(L_QUANTITY) AS AVG_QTY, AVG(L_EXTENDEDPRICE) AS AVG_PRICE, AVG(L_DISCOUNT) AS | |
AVG_DISC, COUNT(*) AS COUNT_ORDER FROM LINEITEM WHERE L_SHIPDATE <= DATE '1998-12-01' | |
- INTERVAL '90' DAY GROUP BY L_RETURNFLAG, L_LINESTATUS ORDER BY L_RETURNFLAG, | |
L_LINESTATUS;' | |
[0.0.1]--- 4 row(s) selected in 21.344s (prep 0.000s, exec 21.344s, 1st fetch 0.000s, | |
fetch 0.000s)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p><code>-q all</code> (or just <code>-q</code>) will not print neither the <strong><em>commands</em></strong> nor the <strong><em>results</em></strong>:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -f script.sql -q all | |
[0.0.0]--- 1 row(s) selected in 0.015s (prep 0.000s, exec 0.015s, 1st fetch -0.000s, | |
fetch -0.000s) | |
[0.0.1]--- 4 row(s) selected in 21.344s (prep 0.000s, exec 21.344s, 1st fetch 0.000s, | |
fetch 0.000s)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>This is often used with odb as query driver.</p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
Even when odb doesn’t print query results (<code>-q res</code>), the result set will be fetched and data is | |
transferred from the database server to the client. In other words, <code>-q res</code> is somehow similar (but | |
not exactly equivalent) to a <code>/dev/null</code> output redirection. | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div class="paragraph"> | |
<p>A special file name you can use with <code>-f</code> is <code>-</code> (dash).</p> | |
</div> | |
<div class="paragraph"> | |
<p>It means: read the script to be executed from the <em>standard input</em>.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>The following command will <em>copy</em> table definitions from one system to | |
another recreating, on the target system, the same table structures as in the source system:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u u1 -p p1 -d SRC -i t:TRAFODION.CIV04 -x "SHOWDDL &1" \ | |
| odb64luo –u u2 –p p2 -d TGT -f -</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_shell_script_here_document_syntax">4.8. Shell Script "here document" Syntax</h3> | |
<div class="paragraph"> | |
<p>Commonly, there’s a need to <em>embed</em> SQL commands in shell scripts.</p> | |
</div> | |
<div class="paragraph"> | |
<p>Use the <code>-f -</code> (read commands from standard input) odb syntax.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">odb64luo -f - <<-EOF 2>&1 | tee -a $\{LOG} | |
drop table &t:TRAFODION.maurizio.ml%; | |
create table ml2 | |
( | |
id integer | |
, fname char(10) | |
, bdate date | |
, lname char(10) default 'Felici' | |
, comment char(20) | |
, city char(10) | |
) no partitions; | |
EOF</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_parallelize_multiple_commands_and_scripts">4.9. Parallelize Multiple Commands and Scripts</h3> | |
<div class="paragraph"> | |
<p>odb uses threads to run multiple commands in parallel. Each command (<code>-x</code>) or | |
script (<code>-f</code>) will be executed, independently from the others, using a different thread.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>Running scripts in parallel.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -x "select count(*) from types" -f script1.sql</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Uses two <em>independent</em> threads executed in parallel. The first thread will run <code>select count(*) from types</code> and the other <code>script1.sql</code>.</p> | |
</div> | |
<div class="paragraph"> | |
<p>You can also run <strong><em>multiple copies</em></strong> of the same command by adding <code><num>:</code> before | |
<code>-x</code> or <code>-f</code> arguments.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The following command runs the instances of <code>select count(*) from types `, five instances | |
of `script1.sql</code> and three instances of <code>script2.sql</code> in parallel using <code>3 + 5 + 3 = 11</code> threads in total:</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>Running eleven commands and scripts in parallel</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -x 3:"select count(*) from types" -f 5:script1.sql \ | |
-f 3:script2.sql -q | |
[1.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[0.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[2.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[4.0.0]--- 1 row(s) selected in 0.001s (prep 0.000s, exec 0.001s, fetch 0.000s/0.000s) | |
[6.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[5.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[3.0.0]--- 1 row(s) selected in 0.001s (prep 0.000s, exec 0.001s, fetch 0.000s/0.000s) | |
[8.0.0]--- 1 row(s) selected in 0.001s (prep 0.000s, exec 0.001s, fetch 0.000s/0.000s) | |
[7.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[9.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[10.0.0]--- 1 row(s) selected in 0.001s prep 0.000s, exec 0.001s, fetch 0.000s/0.000s)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The first number in <code>[<strong>1</strong>.0.0]</code> is the <strong><em>thread ID</em></strong>. Thread IDs are assigned by odb starting from zero.</p> | |
</div> | |
<div class="paragraph"> | |
<p>You can limit the maximum number of threads with <code>-T</code> option.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>The following command runs the same 11 commands/scripts limiting the number of threads (<strong>and ODBC connections</strong>) to 4:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -x 3:"select count(*) from types" -f 5:script1.sql \ | |
-f 3:script2.sql -q -T 4 | |
[1.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[0.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[2.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[1.3.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[2.1.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[0.1.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[3.0.0]--- 1 row(s) selected in 0.001s (prep 0.000s, exec 0.001s, fetch 0.000s/0.000s) | |
[2.2.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[3.1.0]--- 1 row(s) selected in 0.001s (prep 0.000s, exec 0.001s, fetch 0.000s/0.000s) | |
[0.2.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s) | |
[1.2.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, fetch 0.000s/0.000s)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The thread ID is now in the 0-3 range because the 11 <strong><em>executions</em></strong> were <strong><em>queued</em></strong> into four threads. | |
odb offers several alternatives to queue <code>M</code> executions in <code>N(<M)</code> threads. See below.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_limit_number_of_odb_threads">4.10. Limit Number of odb Threads</h3> | |
<div class="paragraph"> | |
<p>By default, odb creates as many threads as the numbers of executions.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The command in the following example creates <code>1 + 3 + 3 = 7</code> threads. Each thread will start its own ODBC connection.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ ./odb64luo -f script1.sql -f 3:script2.sql -x 3:"<mysqlcmd>"</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You can limit the max number of threads using <code>–T</code>.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ ./odb64luo -f script1.sql -f 3:script2.sql -x 3:"<mysqlcmd>" -T 2</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>This command creates just two threads to execute the seven commands/scripts. odb will never create | |
more threads than needed:</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="exampleblock"> | |
<div class="content"> | |
<div class="paragraph"> | |
<p>~/Devel/odb $ ./odb64luo -f 2:script1.sql -f 3:script2.sql -T 8 -c -q | |
odb [main(1017)] - Warning: won’t be created more thread (8) than needed (5).</p> | |
</div> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_change_executions_distributed_across_threads">4.11. Change Executions Distributed Across Threads</h3> | |
<div class="paragraph"> | |
<p>By default, executions are distributed in round-robin across threads.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ ./odb64luo -f script1.sql -f 3:script2.sql -x 3:"<mysqlcmd>" -T 3</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Using the command above, the execution queue will be as follows:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 25%;"> | |
<col style="width: 25%;"> | |
<col style="width: 25%;"> | |
<col style="width: 25%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top"></th> | |
<th class="tableblock halign-left valign-top">Thread 1</th> | |
<th class="tableblock halign-left valign-top">Thread 2</th> | |
<th class="tableblock halign-left valign-top">Thread3</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Third Execution</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>mysqlcmd</code></p></td> | |
<td class="tableblock halign-left valign-top"></td> | |
<td class="tableblock halign-left valign-top"></td> | |
</tr> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Second Execution</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>mysqlcmd</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>mysqlcmd</code></p></td> | |
</tr> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">First Execution</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>This (standard) behavior can be modified using the following options:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>–Z</code> (shuffle): This option <strong><em>randomizes</em></strong> the execution order.</p> | |
</li> | |
<li> | |
<p><strong><em>factor sign</em></strong> with <code>–P</code> option: See <a href="#query_driver_all_scripts_path">Run All Scripts With a Given Path</a>.</p> | |
</li> | |
<li> | |
<p><code>-dlb</code> (Dynamic Load Balancing): See <a href="#concepts_load_balancing">Dynamic Load Balancing</a>.</p> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="concepts_load_balancing">4.12. Dynamic Load Balancing</h3> | |
<div class="paragraph"> | |
<p>As discussed in the previous section, executions are normally <em>pre-assigned</em> to threads using a simple | |
round-robin algorithm. This way, the total elapsed time for each thread depends on the complexity of | |
<strong>its own</strong> <em>executions</em>.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>Suppose you have two threads and two <em>executions</em> per thread:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 33.3333%;"> | |
<col style="width: 33.3333%;"> | |
<col style="width: 33.3334%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top"></th> | |
<th class="tableblock halign-left valign-top">Thread 1</th> | |
<th class="tableblock halign-left valign-top">Thread 2</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Second Execution</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.2</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.2</code></p></td> | |
</tr> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">First Execution</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.3</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.1</code></p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>If thread 2.1 and 2.2 require a very short time to be executed you can have a situation where Thread2 has | |
nothing to do (it will be terminated) while Thread1 is still busy with <strong>its own</strong> Script1.3 and Script1.2.</p> | |
</div> | |
<div class="paragraph"> | |
<p>In some cases, for example during data extractions (see <a href="#load_binary_files">Load Binary Files</a>), you might want to keep all | |
threads busy at any given time. In these cases you can use Dynamic Load Balancing (<code>-dlb</code>). With Dynamic | |
Load Balancing jobs are not <strong>pre-assigned</strong> to threads when odb starts; each thread will pick the | |
next job to run from the job list <em>at run-time</em>.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_use_variables_in_odb_scripts">4.13. Use Variables in odb Scripts</h3> | |
<div class="paragraph"> | |
<p>odb let you to use two kinds of variables:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>Internal Variables</strong> defined through the <code>set param</code> command and identified by the ampersand character;</p> | |
</li> | |
<li> | |
<p><strong>Environment variables</strong> defined at operating system level and identified by a dollar sign;</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>You can mix internal and environment variables in your scripts. If a variable is not expanded to a valid | |
Internal/Environment variable the text will remain unchanged.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat scr.sql set param region1 ASIA | |
-- region1 is defined as an internal odb parameter | |
select * from tpch.region where r_name = '&region1'; | |
-- region2 is defined as an environment variable | |
select * from tpch.region where r_name = '$region2'; | |
-- you can mix internal and environment variables | |
select * from tpch.region where r_name = '$region2' or r_name = '&region1'; | |
-- region3 variable does not exists so it won’t be not expanded | |
select * from tpch.region where r_name = '&region3';</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>After you define <code>region2</code> at operating system level:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ export region2=AMERICA</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Output:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -u mauro -p xx -d pglocal -f scr.sql | |
odb [2011-12-12 08:01:31]: starting (1) ODBC connection(s)... 1 [0.0.0]Executing: | |
'select * from tpch.region where r_name = 'ASIA';' 2,ASIA,ges. thinly even pinto beans ca | |
[0.0.0]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, 1st fetch 0.000s, fetch 0.000s) | |
[0.0.1]Executing: 'select * from tpch.region where r_name = 'AMERICA';' 1,AMERICA,hs use ironic, even requests. s | |
[0.0.1]--- 1 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, 1st fetch 0.000s, fetch 0.000s) | |
[0.0.2]Executing: 'select * from tpch.region where r_name = 'AMERICA' or r_name = 'ASIA';' 1,AMERICA,hs use ironic, | |
even requests.s2,ASIA,ges. thinly even pinto beans ca | |
[0.0.2]--- 2 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, 1st fetch 0.000s, fetch 0.000s) | |
[0.0.3]Executing: 'select * from tpch.region where r_name = '&region3';' | |
[0.0.3]--- 0 row(s) selected in 0.000s (prep 0.000s, exec 0.000s, 1st fetch 0.000s, fetch 0.000s)</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_thread_id_thread_execution_and_script_command">4.14. Thread ID, Thread Execution#, and Script Command#</h3> | |
<div class="paragraph"> | |
<p>Consider a script containing two commands:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/odb $ cat script.sql | |
SELECT COUNT(*) FROM ORDERS; | |
SELECT COUNT(*) FROM SUPPLIER;</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Running this script ten times using two threads yields:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/odb $ ./odb64luo -f 10:script.sql -q -T 2 | |
[0.0.0]--- 1 row(s) selected in 0.102s (prep 0.022s, exec 0.044s, 1st fetch 0.037s, fetch 0.037s) | |
[1.0.0]--- 1 row(s) selected in 0.125s (prep 0.022s, exec 0.068s, 1st fetch 0.036s, fetch 0.036s) | |
[0.0.1]--- 1 row(s) selected in 0.520s (prep 0.022s, exec 0.048s, 1st fetch 0.450s, fetch 0.450s) | |
[1.0.1]--- 1 row(s) selected in 0.564s (prep 0.017s, exec 0.480s, 1st fetch 0.067s, fetch 0.067s) | |
[0.1.0]--- 1 row(s) selected in 0.131s (prep 0.022s, exec 0.060s, 1st fetch 0.048s, fetch 0.048s) | |
[0.1.3]--- 1 row(s) selected in 0.086s (prep 0.022s, exec 0.057s, 1st fetch 0.007s, fetch 0.007s) | |
[1.3.0]--- 1 row(s) selected in 0.136s (prep 0.035s, exec 0.058s, 1st fetch 0.042s, fetch 0.042s) | |
[0.2.0]--- 1 row(s) selected in 0.123s (prep 0.029s, exec 0.068s, 1st fetch 0.026s, fetch 0.026s) | |
[1.3.1]--- 1 row(s) selected in 0.119s (prep 0.016s, exec 0.082s, 1st fetch 0.021s, fetch 0.021s) | |
[0.2.1]--- 1 row(s) selected in 0.089s (prep 0.031s, exec 0.054s, 1st fetch 0.004s, fetch 0.004s) | |
[1.2.0]--- 1 row(s) selected in 0.138s (prep 0.023s, exec 0.041s, 1st fetch 0.073s, fetch 0.073s) | |
[0.3.0]--- 1 row(s) selected in 0.144s (prep 0.038s, exec 0.045s, 1st fetch 0.061s, fetch 0.061s) | |
[1.2.1]--- 1 row(s) selected in 0.127s (prep 0.016s, exec 0.041s, 1st fetch 0.070s, fetch 0.070s) | |
[0.3.1]--- 1 row(s) selected in 0.136s (prep 0.033s, exec 0.056s, 1st fetch 0.048s, fetch 0.048s) | |
[1.3.0]--- 1 row(s) selected in 0.131s (prep 0.023s, exec 0.037s, 1st fetch 0.071s, fetch 0.071s) | |
[0.4.0]--- 1 row(s) selected in 0.111s (prep 0.033s, exec 0.045s, 1st fetch 0.033s, fetch 0.033s) | |
[0.4.1]--- 1 row(s) selected in 0.076s (prep 0.033s, exec 0.037s, 1st fetch 0.005s, fetch 0.006s) | |
[1.3.1]--- 1 row(s) selected in 0.098s (prep 0.016s, exec 0.065s, 1st fetch 0.017s, fetch 0.017s) | |
[1.4.0]--- 1 row(s) selected in 0.133s (prep 0.023s, exec 0.074s, 1st fetch 0.035s, fetch 0.035s) | |
[1.4.1]--- 1 row(s) selected in 0.098s (prep 0.017s, exec 0.064s, 1st fetch 0.016s, fetch 0.016s)</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>The numbers between square brackets have the following meaning:</p> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p>The first digit is the <strong><em>thread ID</em></strong>. The example above has two threads; the ID is either 0 or 1.</p> | |
</li> | |
<li> | |
<p>The second digit is the <strong><em>execution#</em></strong> for a given thread. The example above has ten script executions | |
for two threads, each thread will have to execute this script five times.; <strong><em>execution#</em></strong>, is between 0 and 4.</p> | |
</li> | |
<li> | |
<p>The third (last) digit is the <strong><em>command#</em></strong> in a given script. The script in the example above contains two | |
commands; this value is 0 or 1.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p><code><strong>[0.3.1]</strong></code> means that the <strong>first thread</strong> (<em>thread id=0</em>) was executing its <strong>fourth job</strong> | |
(<em>thread execution#=3</em>) and — more specifically <mark>&8212; the <strong>second command</strong> in that script | |
(<em>script command</mark>=1</em>).</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_validate_sql_scripts">4.15. Validate SQL Scripts</h3> | |
<div class="paragraph"> | |
<p>You can check commands and SQL scripts with odb using the <code>-N</code> (null run) flag. This will just <code>prepare</code> | |
(compile) the commands without executing them and fetching the results.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_different_data_sources_for_different_threads">4.16. Different Data Sources for Different Threads</h3> | |
<div class="paragraph"> | |
<p>Normally all ODBC connections started by odb will use the same Data Source. However, there could be | |
special cases where you want to use different DSN for different threads. In these cases you can use the | |
<code>–ndsn <number></code> option. This will append to the Data Source name specified via <code>–d</code> a suffix from <code>1</code> to <code>-ndsn</code> argument.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo ... –d MYDSN –ndsn 4</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>It will use the following (round-robin) DSN/thread association: <code>MYDSN1</code> for the first thread, <code>MYDSN2</code> for the | |
second thread and so on. The fifth thread (if any) will use <code>MYDSN1</code> again. You can use a sequential | |
DSN/thread association by using a <code>+</code> sign in front of the <code>–ndsn</code> argument.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>If you have 16 threads and <code>–d MYDSN</code>:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 33.3333%;"> | |
<col style="width: 33.3333%;"> | |
<col style="width: 33.3334%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Thread ID</th> | |
<th class="tableblock halign-left valign-top">DSN with <code>–ndsn 8</code></th> | |
<th class="tableblock halign-left valign-top">DSN with <code>–ndsn +8</code></th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>0</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN1</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN1</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN2</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN1</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>2</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN3</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN2</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>3</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN4</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN2</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>4</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN5</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN3</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>5</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN6</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN3</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>6</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN7</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN4</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>7</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN8</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN4</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>8</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN1</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN5</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>9</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN2</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN5</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN3</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN6</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>11</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN4</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN6</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>12</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN5</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN7</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>13</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN6</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN7</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>14</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN7</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN8</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>15</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN8</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>MYDSN8</code></p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>This technique has been used to maximize extraction throughput from a multi-segment Trafodion system. | |
Each (local) Data Source was <strong>linked</strong> to a corresponding remote Data Source extracting its own data through its | |
own network interface card.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_format_query_results">4.17. Format Query Results</h3> | |
<div class="paragraph"> | |
<p>Normally odb prints query results using a very basic CSV format.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo -x "select s_suppkey, s_name, s_phone from tpch.supplier limit 5 | |
1,Supplier#000000001,27-918-335-1736 | |
2,Supplier#000000002,15-679-861-2259 | |
3,Supplier#000000003,11-383-516-1199 | |
4,Supplier#000000004,25-843-787-7479 | |
5,Supplier#000000005,21-151-690-3663</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Adding the option <code>-pad</code> you generates the output in table format:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo -x "select s_suppkey, s_name, s_phone from tpch.supplier limit 5" -pad | |
s_suppkey |s_name |s_phone | |
---------------+-------------------------+--------------- | |
1 |Supplier#000000001 |27-918-335-1736 | |
2 |Supplier#000000002 |15-679-861-2259 | |
3 |Supplier#000000003 |11-383-516-1199 | |
4 |Supplier#000000004 |25-843-787-7479 | |
5 |Supplier#000000005 |21-151-690-3663</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_extract_table_ddl">4.18. Extract Table DDL</h3> | |
<div class="paragraph"> | |
<p>You can extract DDL from one or several tables using either the <code>-i D…</code> or | |
<code>-i U…</code> option.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo -u xxx -p xxx -d traf -i D:TRAFODION.SEABASE.REGIONS | |
odb [2015-04-20 21:25:35]: starting ODBC connection(s)... 0 | |
Connected to Trafodion | |
CREATE TABLE TRAFODION.SEABASE."REGIONS" ( REGION_ID INTEGER NOT NULL | |
, REGION_NAME VARCHAR(25) | |
);</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The <code>&</code> wild card allows you to extract the DDL for multiple objects.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>The following command will extract the DDL for all tables in schema <code>tpch</code> starting with <code>P</code>:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo -u xxx -p xxx -d traf -i D:TRAFODION.TPCH.P% | |
odb [2015-04-20 21:33:43]: starting ODBC connection(s)... 0 | |
Connected to Trafodion | |
CREATE TABLE TRAFODION.TPCH."PART" ( P_PARTKEY BIGINT NOT NULL | |
,P_NAME VARCHAR(55) NOT NULL | |
,P_MFGR CHAR(25) NOT NULL | |
,P_BRAND CHAR(10) NOT NULL | |
,P_TYPE VARCHAR(25) NOT NULL | |
,P_SIZE INTEGER NOT NULL | |
,P_CONTAINER CHAR(10) NOT NULL | |
,P_RETAILPRICE BIGINT NOT NULL | |
,P_COMMENT VARCHAR(23) NOT NULL | |
,PRIMARY KEY (P_PARTKEY) | |
); | |
CREATE TABLE TRAFODION.TPCH."PARTSUPP" ( PS_PARTKEY BIGINT NOT NULL | |
,PS_SUPPKEY BIGINT NOT NULL | |
,PS_AVAILQTY INTEGER NOT NULL | |
,PS_SUPPLYCOST BIGINT NOT NULL | |
,PS_COMMENT VARCHAR(199) NOT NULL | |
,PRIMARY KEY (PS_PARTKEY,PS_SUPPKEY) | |
); | |
odb [2015-04-20 21:33:45]: exiting. Session Elapsed time 2.069 seconds (00:00:02.069)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You should consider possible differences in text column length semantic when porting DDLs from one database to another; | |
some databases use "character oriented" text columns length while others use a "byte oriented" semantic.</p> | |
</div> | |
<div class="paragraph"> | |
<p>You can ask odb to multiply text column length when printing DDL using the switch | |
<code>–U[non-wide_char_multiplier,wide_char_multiplier]</code>.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo -u xxx -p xxx -d traf -i U2,4:TRAFODION.TPCH.P% | |
odb [2015-04-20 21:35:17]: starting ODBC connection(s)... 0 | |
Connected to Trafodion | |
CREATE TABLE TRAFODION.TPCH."PART" ( P_PARTKEY BIGINT NOT NULL | |
,P_NAME VARCHAR(110) NOT NULL | |
,P_MFGR CHAR(50) NOT NULL | |
,P_BRAND CHAR(20) NOT NULL | |
,P_TYPE VARCHAR(50) NOT NULL | |
,P_SIZE INTEGER NOT NULL | |
,P_CONTAINER CHAR(20) NOT NULL | |
,P_RETAILPRICE BIGINT NOT NULL | |
,P_COMMENT VARCHAR(46) NOT NULL | |
,PRIMARY KEY (P_PARTKEY) | |
); | |
CREATE TABLE TRAFODION.TPCH."PARTSUPP" ( PS_PARTKEY BIGINT NOT NULL | |
,PS_SUPPKEY BIGINT NOT NULL | |
,PS_AVAILQTY INTEGER NOT NULL | |
,PS_SUPPLYCOST BIGINT NOT NULL | |
,PS_COMMENT VARCHAR(398) NOT NULL | |
,PRIMARY KEY (PS_PARTKEY,PS_SUPPKEY) | |
); | |
odb [2015-04-20 21:35:18]: exiting. Session Elapsed time 1.620 seconds (00:00:01.620)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The command in the above example multiplies the length of "non-wide" text fields by 2 and | |
the length of wide text fields by 4.</p> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_load_extract_copy">5. Load, Extract, Copy</h2> | |
<div class="sectionbody"> | |
<div class="sect2"> | |
<h3 id="_load_files">5.1. Load Files</h3> | |
<div class="paragraph"> | |
<p>You can load a data file using <code>-l</code> option.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn –l src=customer.tbl:tgt=TRAFODION.MAURIZIO.CUSTOMER \ | |
:fs=\|:rows=1000:loadcmd=UL:truncate:parallel=4</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>This command:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Loads the file named <code>customer.tbl</code> (<code>src=customer.tbl</code>)</p> | |
</li> | |
<li> | |
<p>in the table <code>TRAFODION.MAURIZIO.CUSTOMER</code> (<code>tgt=TRAFODION.MAURIZIO.CUSTOMER</code>)</p> | |
</li> | |
<li> | |
<p>using <code>|</code> (vertical bar) as a field separator (<code>fs=\|</code>)</p> | |
</li> | |
<li> | |
<p>using <code>1000 rows</code> as row-set buffer (<code>rows=1000</code>)</p> | |
</li> | |
<li> | |
<p>using <code>UPSERT USING LOAD</code> syntax to achieve better throughput as described in | |
<a href="http://trafodion.apache.org/docs/load_transform/index.html">Trafodion Load and Transform Guide</a></p> | |
</li> | |
<li> | |
<p>truncating the target table before loading (<code>truncate</code>)</p> | |
</li> | |
<li> | |
<p>using <code>4 parallel threads</code> to load the target table (<code>parallel=4</code>)</p> | |
</li> | |
</ul> | |
</div> | |
<div class="sect3"> | |
<h4 id="_data_loading_operators">5.1.1. Data Loading Operators</h4> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">-l src=[-]file:tgt=table[:map=mapfile][:fs=fieldsep][:rs=recsep][:soe] | |
[:skip=linestoskip][:ns=nullstring][:ec=eschar][:sq=stringqualifier] | |
[:pc=padchar][:em=embedchar][:errmax=#max_err][:commit=auto|end|#rows|x#rs] | |
[:rows=#rowset][:norb][:full][:max=#max_rec][:truncate][:show][:bpc=#][:bpwc=#] | |
[:nomark][:parallel=number][:iobuff=#size][:buffsz=#size]][:fieldtrunc=\{0-4}] | |
[:pre=\{@sqlfile}|\{[sqlcmd]}][:post=\{@sqlfile}|\{[sqlcmd]}][:ifempty] | |
[:direct][:bad=[+]badfile][:tpar=#tables][:maxlen=#bytes][:time] | |
[:xmltag=[+]element][:xmlord][:xmldump][:loadcmd=IN|UP|UL]</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>The following table describes each data loading operator:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 35%;"> | |
<col style="width: 65%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Load option</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>src=<file></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Input file. You can use the following keywords for this field:<br> | |
<br> | |
- <code>%t</code> expand to the (lower case) table name<br> | |
- <code>%T</code> expand to the (upper case) table name<br> | |
- <code>%s/%S</code> expand to the schema name<br> | |
- <code>%c/%C</code> expand to the catalog name<br> | |
- <code>stdin</code> load reading from the standard input<br> | |
- <code>-<file></code> to load all files listed in <code><file></code><br> | |
- <code>[hdfs][@host,port[,user]].<hdfspath></code> to load files from Hadoop File System (via <code>libhdfs.so</code>)<br> | |
- <code>[mapr][@host,port[,user]].<maprpath></code> to load files from MapR File System (via <code>libMapRClient.so</code>)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tgt=<CAT.SCH.TAB></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the target table</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>fs=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the field separator. You can define the field separator:<br> | |
<br> | |
- as normal character (for example <code>fs=,</code>)<br> | |
- as ASCII decimal (for example <code>fs=44</code> — <code>44</code> means comma)<br> | |
- as ASCII octal value (for example <code>fs=054</code> — <code>054</code> means comma)<br> | |
- as ASCII hex value (for example <code>fs=x2C</code> — <code>x2C</code> means comma)<br> | |
<br> | |
Default field separator is <code>,</code> (comma)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rs=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the record separator. You can define the record separator the | |
same way as the field separator. Default record separator is <code>\n</code> (new line)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pc=<char|code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Pad character used when loading fixed format files. You can use the same | |
notation as the field separator.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>map=<mapfile></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Uses mapfile to map source file to target table columns. See <a href="#load_map_fields">Map Source File Fields to Target Table Columns</a>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>skip=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Skips a given number of lines when loading. This can be useful to skip headers in the source file.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>max=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The max number of records to load. Default is to load all records in the input file</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ns=<nullstring></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb inserts NULL when it finds nullstring in the input file. By default the nullstring is the empty string</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>sq=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The string qualifier character used to enclose strings. You can define the escape character the same way as the field separator.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ec=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The character used as escape character. You can define the escape character the same way as the field separator.<br> | |
<br> | |
Default is <code>\</code> (back slash).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rows=<num>|k<num>|m<num></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This defines the size of the I/O buffer for each loading thread.<br> | |
<br> | |
You can define the size of this buffer in two different ways:<br> | |
<br> | |
1. number of rows (for example: <code>rows=100</code> means 100 rows as IO buffer)<br> | |
2.* buffer size in KB or MB (for example: <code>rows=k512</code> (512 KB buffer) or <code>rows=m20</code> (20MB buffer))<br> | |
<br> | |
Default value is <code>100</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>bad=[+]file</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Where to write rejected rows. If you omit this parameter, then rejected rows is printed to standard error together with the error returned by | |
the ODBC Driver.<br> | |
<br> | |
If you add a <code>+</code> sign in front of the file-name, odb <strong>appends</strong> to <code><file>`instead of <strong>create</strong> the `<file></code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>truncate</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Truncates the target table before loading.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ifempty</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Loads the target table only if it contains no records.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>norb</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Loads <code>WITH NO ROLLBACK</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nomark</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Don’t print the number of records loaded so far during loads.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>soe</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Stop On Error — stop as soon as odb encounters an error.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>parallel=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Number of loading threads. odb uses:<br> | |
<br> | |
- one thread to read from the input file and<br> | |
- as many threads as the parallel argument to write via ODBC. This option is database independent.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>errmax=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb prints up to num error messages per rowset. Normally used with soe to limit the number of error messages printed to the standard error stream.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>commit=auto|end|#rows|x#rs</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines how odb commits the inserts. You have the following choices:<br> | |
<br> | |
- <code>auto</code> (default): Commit every single insert (see also rows load operator).<br> | |
- <code>end</code>: Commit when all rows (assigned to a given thread) have been inserted.<br> | |
- <code>#rows</code>: Commit every <code>#rows</code> inserted rows.<br> | |
- <code>x#rs</code>: Commit every <code>#rs</code> rowset (see <code>rows</code>)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>direct</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Adds <code>/*+ DIRECT */</code>* hint to the insert statement. To be used with Vertica databases in order to store inserted rows <strong>directly</strong> into | |
the Read-Only Storage (ROS). See Vertica’s documentation.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>fieldtrunc={0-4}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines how odb manages fields longer than the destination target column:<br> | |
<br> | |
- <code>fieldtrunc=0</code> (default): Truncates input string, print a warning and load the truncated field if the target column is a text field.<br> | |
- <code>fieldtrunc=1</code>: Like <code>fieldtrunc=0</code> but no warning message is printed.<br> | |
- <code>fieldtrunc=2</code>: Prints an error message and does NOT load the row.<br> | |
- <code>fieldtrunc=3</code>: Like <code>fieldtrunc=0</code> but tries to load the field even if the target column is NOT a text field.<br> | |
- <code>fieldtrunc=4</code>: Like fieldtrunc=3 but no warnings are printed.<br> | |
<br> | |
WARNING: the last two options could bring to unwanted results. For example, an input string like <code>2001-10-2345</code> is loaded as a valid | |
2001-10-23 if the target field is a <code>DATE</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>em=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Character used to embed binary files. See <a href="#load_default_values">Load Default Values</a>. You can define | |
the embed character the same way as the field separator. No default value.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pre={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb runs a <strong>single instance</strong> of either <code>sqlfile</code> script or <code>sqlcmd</code> (enclosed between square brackets) | |
on the <strong>target system</strong> immediately before loading the target table. You can, for example, CREATE the target table before loading it.<br> | |
<br> | |
Target table is not loaded if SQL execution fails and <code>Stop On Error (soe)</code> is set.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>post={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb runs a <strong>single instance</strong> of either <code>sqlfile</code> script or <code>sqlcmd</code> (enclosed between square brackets) | |
on the <strong>target system</strong> immediately after the target table has been loaded. You can, for example, update database stats after loading a table.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpar=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb loads <code>num</code> tables in parallel when <code>src</code> is a list of files to be loaded.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>show</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb prints what would be loaded in each column but no data is actually loaded. This is useful if you want to see how the input file | |
<em>fits</em> into the target tables, Normally used to analyze the first few rows of CSV files (use <code>:max</code>). This option forces:<br> | |
<br> | |
- <code>parallel</code> to <code>1</code>.<br> | |
- <code>rows</code> to <code>1</code>.<br> | |
- <code>ifempty</code> to <code>false</code>.<br> | |
- <code>truncate</code> to <code>false</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>maxlen=#bytes</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb limits the amount of memory allocated in the ODBC buffers for CHAR/VARCHAR fields to <code>#bytes</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>time</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb prints a <strong>time line</strong> (milliseconds from start) for each insert.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>bpc=#</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Bytes allocated in the ODBC buffer for each (non wide) CHAR/VARCHAR column length unit. (Default: 1)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>bwpc=#</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Bytes allocated in the ODBC buffer for each (wide) CHAR/VARCHAR column length unit. (Default: 4)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Xmltag=[+]tag</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Input file is XML. Load all <em>XML nodes</em> under the one specified with this option. If a plus sign is | |
specified, then odb loads node-attributes values.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>xmlord</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">By default, odb <em>matches</em> target table columns with XML node or attributes using their names. If this option is specified, then | |
odb loads the first node/attribute to the first column, the second node/attribute to the second column and so on without checking node/attribute names.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>xmldump</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb does not load the XML file content. Instead, XML attribute/tage names are printed to standard output so you can check | |
what is going to be loaded.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>loadcmd</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">SQL operation to be used for load. (Default: <code>INSERT</code>). <code>UPSERT</code> and <code>UPSERT USING LOAD</code> are also available for Trafodion.</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>You can load multiple files using different <code>-l</code> options. By default odb creates as many threads (and ODBC connections) as the sum of | |
parallel load threads. You can limit this number using <code>-T</code> option.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn –T 5 \ | |
-l src=./data/%t.tbl.gz:tgt=TRAFODION.MAURO.CUSTOMER:fs=\ | |
|:rows=m2:truncate:norb:parallel=4 \ | |
-l src=./data/%t.tbl.gz:tgt=TRAFODION.MAURO.ORDERS:fs=\ | |
|:rows=1000:truncate:norb:parallel=4 \ | |
-l src=./data/%t.tbl.gz:tgt=TRAFODION.MAURO.LINEITEM:fs=\ | |
|:rows=m10:truncate:norb:parallel=4</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The above command truncates and loads the CUSTOMER, ORDERS and LINEITEM tables. The input files have the same name as the | |
target tables -– in lower case). Loads are distributed among available threads this way:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 10%;"> | |
<col style="width: 18%;"> | |
<col style="width: 18%;"> | |
<col style="width: 18%;"> | |
<col style="width: 18%;"> | |
<col style="width: 18%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Load Order</th> | |
<th class="tableblock halign-left valign-top">Thread 0</th> | |
<th class="tableblock halign-left valign-top">Thread 1</th> | |
<th class="tableblock halign-left valign-top">Thread2</th> | |
<th class="tableblock halign-left valign-top">Thread3</th> | |
<th class="tableblock halign-left valign-top">Thread4</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Third</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Read <code>lineitem.tbl</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.LINEITEM</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.LINEITEM</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.LINEITEM</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.LINEITEM</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Second</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Read <code>orders.tbl</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.ORDERS</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.ORDERS</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.ORDERS</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.ORDERS</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">First</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Read <code>customer.tbl</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.CUSTOMER</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.CUSTOMER</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.CUSTOMER</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Load <code>TRAFODION.MAURO.CUSTOMER</code></p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>If you want to load more than one table in parallel you should use a number of threads defined as: | |
<code>(parallel + 1) * tables_to_load_in_parallel</code></p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
You can load gzipped files without any special option. odb automatically checks input files and decompress them on the fly when | |
needed. | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div class="paragraph"> | |
<p>odb using one single loading thread (<code>parallel=1</code>) is faster than without parallel — if you do not specify | |
parallel, odb uses one thread to both read from file and write into the target table:</p> | |
</div> | |
<div class="paragraph"> | |
<p><code>Read buffer #1>Write Buffer #1>Read Buffer #2>Write Buffer #2>Read Buffer #3>Write Buffer#3>…</code></p> | |
</div> | |
<div class="paragraph"> | |
<p><code>parallel=1</code> defines that there is one thread to read from file and one thread to write:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>Read buffer #1>Read Buffer #2>Read Buffer #3>…</code></p> | |
</li> | |
<li> | |
<p><code>Write Buffer #1>Write Buffer #2>Write Buffer #3>…</code></p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>Reading from file is <strong>normally</strong> much faster than writing via ODBC so a single <em>reading thread</em> can serve different <em>loading threads</em>. | |
One could ask: what the <em>right</em> number of loading threads is?</p> | |
</div> | |
<div class="paragraph"> | |
<p>In order to define the right number of loading threads you should run a few test and monitor the <em>Wait Cycles</em> reported by odb. | |
Wait Cycles represent the number of times the <em>reading thread</em> had to wait for one <em>loading thread</em> to become available.</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>When you have high <em>Wait Cycles/Total Cycles” ratio…</em> it’s better to increase the number of writers.</p> | |
</li> | |
<li> | |
<p>When the <em>Wait Cycles/Total Cycles</em> is less than 5%, adding more loading threads is useless or counterproductive.</p> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="load_map_fields">5.2. Map Source File Fields to Target Table Columns</h3> | |
<div class="paragraph"> | |
<p>odb, <em>by default</em>, assumes that input files contain as many fields as the target table columns, and that file fields and target | |
table columns are in the same order. This means that the first field in the input file is loaded in the first table column, | |
second input field goes to the second column and so on.</p> | |
</div> | |
<div class="paragraph"> | |
<p>If this basic assumption is not true and you need more flexibility to <em>link</em> input fields to target table columns, then | |
odb provides mapping/transformation capabilities though <strong>mapfiles</strong>. By specifying <code>map=<mapfile></code> load option you can:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Associate any input file field to any table column</p> | |
</li> | |
<li> | |
<p>Skip input file fields</p> | |
</li> | |
<li> | |
<p>Generate sequences</p> | |
</li> | |
<li> | |
<p>Insert constants</p> | |
</li> | |
<li> | |
<p>Transform dates/timestamps formats</p> | |
</li> | |
<li> | |
<p>Extract substrings</p> | |
</li> | |
<li> | |
<p>Replace input file strings. For example: insert <code>Maurizio Felici</code> when you read <code>MF</code></p> | |
</li> | |
<li> | |
<p>Generate random values</p> | |
</li> | |
<li> | |
<p>… and much more</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>A generic <em>mapfile</em> contains:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>Comments</strong> (line starting with <code>#</code>)</p> | |
</li> | |
<li> | |
<p><strong>Mappings</strong> to link input file fields to the corresponding target table columns.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>Mappings use the following syntax:</p> | |
</div> | |
<div class="paragraph"> | |
<p><code><colname>:<field>[:transformation operator]</code></p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>Where:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code><colname></code> is the target table column name. (Case sensitive)</p> | |
</li> | |
<li> | |
<p><code><field></code> is one of the following:</p> | |
</li> | |
<li> | |
<p>The ordinal position (<code><em>starting from zero</em></code>) of the input file field.</p> | |
<div class="paragraph"> | |
<p>First input field is <code>0</code> (zero), second input field is <code>1</code> and so on</p> | |
</div> | |
</li> | |
<li> | |
<p><code>CONST:<CONSTANT></code> to load a constant value</p> | |
</li> | |
<li> | |
<p><code>SEQ:<START></code> to generate/load a sequence starting from <code><START></code></p> | |
</li> | |
<li> | |
<p><code>IRAND:<MIN>:<MAX></code> to generate/load a random integer between <code><MIN></code> and <code><MAX></code></p> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>DRAND:<MIN_YEAR>:<MAX_YEAR></code> to generate/load a random date (<code>YYYY-MM-DD</code>) between <code><MIN_YEAR></code> and <code><MAX_YEAR></code></p> | |
</li> | |
<li> | |
<p><code>TMRAND</code>: to generate/load a random time (<code>hh:mm:ss</code>) between <code>00:00:00</code> and <code>23:59:59</code></p> | |
</li> | |
<li> | |
<p><code>TSRAND</code>: to generate/load a random timestamp (<code>YYYY-MM-DD hh:mm:ss</code>) between midnight UTC –- 01 Jan 1970 and the current timestamp</p> | |
</li> | |
<li> | |
<p><code>CRAND:<LENGTH></code> generates/loads a string of <code><LENGTH></code> characters randomly selected in the following ranges: <code>a-z</code>, <code>A-Z</code>, <code>0-9</code></p> | |
</li> | |
<li> | |
<p><code>NRAND:<PREC>:<SCALE></code> generates/loads a random NUMERIC field with precision <code><PREC></code> and scale <code><SCALE></code></p> | |
</li> | |
<li> | |
<p><code>DSRAND:<file></code> selects and loads a random line from <code><file></code></p> | |
</li> | |
<li> | |
<p><code>TXTRAND:<MIN_LENGTH>:<MAX_LENGTH>:<file>:</code> selects and loads a random portion of test from <code><file></code> with length between <code><MIN_LENGTH></code> and <code><MAX_LENGTH></code></p> | |
</li> | |
<li> | |
<p><code>LSTRAND:<VALUE1,VALUE2,…></code> selects and loads a random value from <code><VALUE1,VALUE2,…></code></p> | |
</li> | |
<li> | |
<p><code>EMRAND:<MIN_ULENGTH>:<MAX_ULENGTH>:<MIN_DLENGTH>:<MAX_DLENGTH>:<SUFFIX1,SUFFIX2,…></code> generates and loads a string made of <code>local@domain.suffix</code> where:</p> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>local is a string of random characters (<code>a-z</code>, <code>A-Z</code>, <code>0-9</code>) with length between <code><MIN_ULENGTH></code> and <code><MAX_ULENGTH></code></p> | |
</li> | |
<li> | |
<p>domain is a string of random characters (<code>a-z</code>, <code>A-Z</code>, <code>0-9</code>) with length between <code><MIN_DLENGTH></code> and <code><MAX_DLENGTH></code></p> | |
</li> | |
<li> | |
<p>suffix is a randomly selected suffix from <code><SUFFIX1,SUFFIX2,…></code></p> | |
</li> | |
</ul> | |
</div> | |
</li> | |
<li> | |
<p><code>CDATE</code>: to load the current date (<code>YYYY-MM-DD</code>)</p> | |
</li> | |
<li> | |
<p><code>CTIME</code>: to load the current time (<code>hh:mm:ss</code>)</p> | |
</li> | |
<li> | |
<p><code>CTSTAMP</code>: to load the current timestamp (<code>YYYY-MM-SS hh:mm:ss</code>)</p> | |
</li> | |
<li> | |
<p><code>FIXED:<START>:<LENGTH></code> to load fixed format fields made of <code><LENGTH></code> characters starting at <code><START></code>.</p> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
<code><START></code> starts from zero. | |
</td> | |
</tr> | |
</table> | |
</div> | |
</li> | |
<li> | |
<p><code>EMPTYASNULL</code>: loads empty strings in the input file as NULLs (default is to load empty string as empty strings)</p> | |
</li> | |
<li> | |
<p><code>EMPTYASCONST:<CONSTANT></code>: loads empty fields in the input file as <code><CONSTANT></code></p> | |
</li> | |
<li> | |
<p><code>NULL</code>: inserts <code>NULL</code></p> | |
</li> | |
<li> | |
<p><code>:transformation operators</code> (optional):</p> | |
</li> | |
<li> | |
<p><code>SUBSTR:<START>:<END></code>. For example, if you have an input field containing <code>Tib:student</code> a transformation rule | |
like <code>SUBSTR:3:6`m then `Tib</code> is loaded into the database.</p> | |
</li> | |
<li> | |
<p><code>TSCONV:<FORMAT></code>. Converts timestamps from the input file format defined through <code><FORMAT></code> to | |
<code>YYYY-MM-DD HH:MM:SS</code> before loading. The input format is defined through any combination of the following characters:</p> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 15%;"> | |
<col style="width: 85%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Char</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>b</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">abbreviated month name</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>B</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">full month name</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>d</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">day of the month</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>H</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">hour (24 hour format)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>m</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">month number</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>M</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Minute</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>S</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Second</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>y</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">year (four digits)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>D#</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">#decimal digits</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>.</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">ignore a single char</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>_</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">ignore up to the next digit</p></td> | |
</tr> | |
</tbody> | |
</table> | |
</li> | |
<li> | |
<p><code>DCONV:<FORMAT></code>. Converts dates from the input file format defined through <code><FORMAT></code> to <code>YYYY-MM-DD</code> (see <code>TSCONV</code> operator).<br> | |
<br> | |
Example: <code>DCONV:B.d.y</code> converts <code>August,24 1991</code> to <code>1991-08-24</code></p> | |
</li> | |
<li> | |
<p><code>TCONV:<FORMAT></code>. Converts times from the input file format defined through <code><FORMAT></code> to <code>HH:MM:SS</code> (see <code>TSCONV</code> operator).</p> | |
</li> | |
<li> | |
<p><code>REPLACE:<READ>:<WRITTEN></code>. Loads the string <code><WRITTEN></code> when the input file fields contains <code><READ></code>. | |
If the input file string doesn’t match <code><READ></code>, then it is loaded as is.</p> | |
<div class="paragraph"> | |
<p>See <a href="#load_mapfiles_ignore">Use mapfiles to Ignore and/or Transform Fields When Loading</a></p> | |
</div> | |
</li> | |
<li> | |
<p><code>TOUPPER</code>. Converts the string read from the input file to uppercase before loading.</p> | |
<div class="paragraph"> | |
<p>Example: <code>proGRAmMEr —> PROGRAMMER</code></p> | |
</div> | |
</li> | |
<li> | |
<p><code>TOLOWER</code>. Converts the string read from the input file to lowercase before loading.</p> | |
<div class="paragraph"> | |
<p>Example: <code>proGRAmMEr —> programmer</code></p> | |
</div> | |
</li> | |
<li> | |
<p><code>FIRSTUP</code>. Converts the first character of the string read from the input file to uppercase and | |
the remaining characters to lowercase before loading.</p> | |
<div class="paragraph"> | |
<p>Example: <code>proGRAmMEr —> Programmer</code></p> | |
</div> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>TRANSLIT:<LIST OF CHARS>:<LIST OF CHARS></code>. Lets you to delete or change any character with another.</p> | |
<div class="paragraph"> | |
<p><strong>Examples</strong></p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>WORK:7:translit:Gp:HP</code> loads the seventh input field into the target column named <code>WORK</code> and replaces all <code>G</code> | |
with <code>H</code> and all <code>p</code> with <code>P</code></p> | |
</li> | |
<li> | |
<p><code>WORK:7:translit:Gp\r:HP\d</code> behaves like the previous example but also deletes all <code>carriage returns</code> (<code>\r</code>)</p> | |
</li> | |
<li> | |
<p><code>CSUBSTR</code>. This operator is somehow similar to <code>SUBSTR</code> but instead of using fixed position to extract substrings | |
will use delimiting characters. For example, suppose your input fields (comma is the field separator) are:</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">... other fields...,name_Maurizio.programmer,...other fields | |
... other fields...,_name_Lucia.housewife, ...other fields... | |
... other fields...,first_name_Giovanni.farmer,... other fields... | |
... other fields...,_Antonella,... other fields... | |
... other fields...,Martina,...other fields... | |
... other fields...,Marco.student, ...other fields...</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Using a transformation like: <code>NAME:4:CSUBSTR:95:46</code> (where <code>95</code> is the ASCII code for <code>_</code> and 46 is the ASCII code for <code>.</code>) | |
results in loading the following values into the target (<code>NAME</code>) column:<br></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">Maurizio | |
Lucia | |
Giovanni | |
Antonella | |
Martina | |
Marco</code></pre> | |
</div> | |
</div> | |
</li> | |
</ul> | |
</div> | |
</li> | |
<li> | |
<p><code>COMP</code>. Transform a packed binary <code>COMP</code> into a target database number.</p> | |
<div class="paragraph"> | |
<p>For example: <code>hex 80 00 00 7b</code> is loaded as <code>-123</code></p> | |
</div> | |
</li> | |
<li> | |
<p><code>COMP3:PRECISION:SCALE</code>. Transform a packed binary <code>COMP-3</code> format into a target database number.</p> | |
<div class="paragraph"> | |
<p>For example: <code>hex 12 34 56 78 90 12 34 56 78 9b</code> is loaded as <code>-1234567890123456.789</code></p> | |
</div> | |
</li> | |
<li> | |
<p><code>ZONED:PRECISION:SCALE</code>. Transform a packed binary <code>ZONED</code> format into a target database number.</p> | |
<div class="paragraph"> | |
<p>For example: <code>hex 31 32 33 34 35 36</code> is loaded as <code>+.123456</code></p> | |
</div> | |
</li> | |
</ul> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="load_mapfiles_ignore">5.3. Use mapfiles to Ignore and/or Transform Fields When Loading</h3> | |
<div class="paragraph"> | |
<p>The following example explains mapfile usage to skip/transform or generate fields. Suppose you have a target table like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">+------+---------------+----+-------+------------+ | |
|COLUMN|TYPE |NULL|DEFAULT|INDEX | | |
+------+---------------+----+-------+------------+ | |
|ID |INTEGER SIGNED |NO | |mf_pkey 1 U | | |
|NAME |CHAR(10) |YES | | | | |
|AGE |SMALLINT SIGNED|YES | | | | |
|BDATE |DATE |YES | | | | |
+------+---------------+----+-------+------------+</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>And an input file like this:</p> | |
</div> | |
<hr> | |
<div class="paragraph"> | |
<p>uno,00,<strong>51</strong>,due,<em>Maurizio</em>,tre,<span class="underline">07 Mar 1959</span>, ignore,remaining, fields<br> | |
uno,00,<strong>46</strong>,due,<em>Lucia</em>,tre,<span class="underline">13 Oct 1964</span>, ignore, this<br> | |
uno,00,<strong>34</strong>,due,<em>Giovanni</em>,tre,<span class="underline">30 Mar 1976</span><br> | |
uno,00,<strong>48</strong>,due,<em>Antonella</em>,tre,<span class="underline">24 Apr 1962</span> | |
<strong>*</strong></p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>Bold text</strong> represents age.</p> | |
</li> | |
<li> | |
<p><em>Italics text</em> represents name.</p> | |
</li> | |
<li> | |
<p><span class="underline">Underline text</span> represents birth date.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>You want to load the marked fields into the appropriate column, <strong><em>generate</em></strong> a unique key for ID and ignore the remaining fields. | |
In addition: you need to <strong><em>convert date format</em></strong> and replace all occurrences of <code>Lucia</code> with <code>Lucy</code>.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The following map file accomplishes these goals:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ cat test/load_map/ml1.map | |
# Map file to load TRAFODION.MFTEST.FRIENDS from friends.dat | |
ID:seq:1 # Inserts into ID column a sequence starting from 1 | |
NAME:4:REPLACE:Lucia:Lucy # Loads field #4 into NAME and replace all occurrences of Lucia with Lucy | |
AGE:2 # Loads field #2 (they start from zero) into AGE | |
BDATE:6:DCONV:d.b.y # Loads field #6 into BDATE converting date format from dd mmm yyyy</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>Load as follows:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn \ | |
-l src=friends.dat:tgt=TRAFODION.MFTEST.FRIENDS:map=ml1.map:fs=,</code></pre> | |
</div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_use_mapfiles_to_load_fixed_format_files">5.4. Use mapfiles to Load Fixed Format Files</h3> | |
<div class="paragraph"> | |
<p>Suppose you have a target table like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">+------+---------------+----+-------+------------+ | |
|COLUMN|TYPE |NULL|DEFAULT|INDEX | | |
+------+---------------+----+-------+------------+ | |
|NAME |CHAR(10) |YES | | | | |
|JOB |CHAR(10) |YES | | | | |
|BDATE |DATE |YES | | | | |
+------+---------------+----+-------+------------+</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>And an input file like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">GiovanniXXX30 Mar 1976YFarmer | |
Lucia XXX13 Oct 1964YHousewife | |
Martina XXX28 Oct 1991Y? | |
Marco XXX06 Nov 1994Y? | |
MaurizioXXX07 Mar 1959YProgrammer</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You want to load the fixed-position fields into the appropriate columns and to <strong><em>convert date format</em></strong>. | |
Null values in the input file are represented by question marks. In this case you can use a mapfile like | |
this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat test/fixed/ff.map | |
NAME:FIXED:0:8 <- insert into NAME characters starting at position 0, length 8 | |
BDATE:FIXED:11:11:DCONV:d.b.y <- insert into BDATE characters starting at col 11, length 11 and convert date | |
JOB:FIXED:23:10 <- insert into JOB characters starting at position 23, length 10</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>Load as follows:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn \ | |
-l src=frends1.dat:tgt=TRAFODION.MFTEST.FRIENDS1:map=ff.map:ns=\?:pc=32</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Where: <code>pc=32</code> identify the pad character in the input file (<code>space</code> = ASCII 32) and <code>ns=?</code> defines | |
the null string in the input file.</p> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_generate_and_load_data">5.5. Generate and Load Data</h3> | |
<div class="paragraph"> | |
<p>odb can generate and load data for testing purposes. The following example illustrates | |
the odb capabilities in this area through an example.</p> | |
</div> | |
<div class="paragraph"> | |
<p>Suppose you want to fill with test data a table like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">CREATE TABLE TRAFODION.MAURIZIO."PERSON" | |
( PID BIGINT SIGNED NOT NULL | |
, FNAME CHAR(20) NOT NULL | |
, LNAME CHAR(20) NOT NULL | |
, COUNTRY VARCHAR(40) NOT NULL | |
, CITY VARCHAR(40) NOT NULL | |
, BDATE DATE NOT NULL | |
, SEX CHAR(1) NOT NULL | |
, EMAIL VARCHAR(40) NOT NULL | |
, SALARY NUMERIC(9,2) NOT NULL | |
, EMPL VARCHAR(40) NOT NULL | |
, NOTES VARCHAR(80) | |
, LOADTS TIMESTAMP(0) | |
, PRIMARY KEY (PID) | |
) | |
;</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>You can use a mapfile like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat person.map | |
PID:SEQ:100 | |
FNAME:DSRAND:datasets/first_names.txt | |
LNAME:DSRAND:datasets/last_names.txt | |
COUNTRY:DSRAND:datasets/countries.txt | |
CITY:DSRAND:datasets/cities.txt | |
BDATE:DRAND:1800:2012 | |
SEX:LSTRAND:M,F,U | |
EMAIL:EMRAND:3:12:5:8:com,edu,org,net | |
SALARY:NRAND:9:2 | |
EMPL:DSRAND:datasets/fortune500.txt | |
NOTES:TXTRAND:20:80:datasets/lorem_ipsum.txt | |
LOADTS:CTSTAMP</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Where:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code>PID:SEQ:100</code> — Loads a sequence starting from <code>100</code> into <code>PID</code></p> | |
</li> | |
<li> | |
<p><code>FNAME:DSRAND:datasets/first_names.txt</code> — Loads <code>FNAME</code> with a randomly selected value from <code>first_names.txt</code>. | |
There are plenty of sample datasets available to generate all sort of data using <em>realistic</em> values.</p> | |
</li> | |
<li> | |
<p><code>LNAME:DSRAND:datasets/last_names.txt</code> — Loads <code>LNAME</code> with a random value from <code>last_names.txt</code>.</p> | |
</li> | |
<li> | |
<p><code>COUNTRY:DSRAND:datasets/countries.txt</code> — Loads <code>COUNTRY</code> with a random value from <code>countries.txt</code>.</p> | |
</li> | |
<li> | |
<p><code>CITY:DSRAND:datasets/cities.txt</code> — Loads <code>CITY</code> with a random value from <code>cities.txt</code>.</p> | |
</li> | |
<li> | |
<p><code>BDATE:DRAND:1800:2012</code> — Generates and loads into <code>BDATE</code> a random date between <code>1800-01-01</code> and <code>2012-12-31</code>.</p> | |
</li> | |
<li> | |
<p><code>SEX:LSTRAND:M,F,U</code> — Loads <code>SEX</code> with a random value in the <code>M</code>, <code>F</code>, <code>U</code> range.</p> | |
</li> | |
<li> | |
<p><code>EMAIL:EMRAND:3:12:5:8:com,edu,org,net</code> — Generates and loads a <code>local@domain.suffix email</code> addresses where:</p> | |
</li> | |
<li> | |
<p><code>local</code> is made of 3 to 12 random characters.</p> | |
</li> | |
<li> | |
<p><code>domain</code> is made of 5 to 8 random characters.</p> | |
</li> | |
<li> | |
<p><code>suffix</code> is <code>com</code>, <code>ord</code>, <code>edu</code>, or <code>net</code>.</p> | |
</li> | |
<li> | |
<p><code>SALARY:NRAND:9:2</code> — Generate and loads a random NUMERIC(9,2).</p> | |
</li> | |
<li> | |
<p><code>EMPL:DSRAND:datasets/fortune500.txt</code> — Loads <code>EMPL</code> with a random value from <code>fortune500.txt</code>.</p> | |
</li> | |
<li> | |
<p><code>NOTES:TXTRAND:20:80:datasets/lorem_ipsum.txt</code> — Loads <code>NOTES</code> with a random section of <code>lorem_ipsum.txt</code> | |
with length between 20 and 80 characters`</p> | |
</li> | |
<li> | |
<p><code>LOADTS:CTSTAMP</code> — Loads the current timestamp into <code>LOADTS</code>.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>You generate and load test data with a command like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ bin/odb64luo -u user -p password -d traf -l src=nofile: | |
tgt=traf.maurizio.person:max=1000000: | |
map=person.map:rows=5000:parallel=8:loadcmd=INSERT</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Please note <code>src=nofile” (it means <em>there is no input file</em>) and `max=1000000</code> (generate and load one million rows). The above command | |
has generated and loaded 1M rows of <em>realistic</em> data in about ten seconds:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">[0] odb Loading statistics: | |
[0] Target table: TRAFODION.MAURIZIO.PERSON | |
[0] Source: nofile | |
[0] Pre-loading time: 2.911 s | |
[0] Loading time: 7.466 s | |
[0] Total records read: 1,000,000 | |
[0] Total records inserted: 1,000,000 | |
[0] Total number of columns: 12 | |
[0] Total bytes read: 3,963 | |
[0] Average input row size: 0.0 B | |
[0] ODBC row size: 323 B (data) + 88 B (len ind) [0] Rowset size: 5,000 | |
[0] Rowset buffer size: 2,006.83 KiB | |
[0] Load Performances (real data): 0.518 KiB/s | |
[0] Load Performances(ODBC): 42,243.161 KiB/s | |
[0] Reader Total/Wait Cycles: 200/16</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="load_default_values">5.6. Load Default Values</h3> | |
<div class="paragraph"> | |
<p>The simpler way to load database generated defaults is to ignore the associated columns in the map file. For example, suppose you have a | |
table like this under Trafodion:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">create table TRAFODION.maurizio.dtest | |
( id largeint generated by default as identity not null | |
, fname char(10) | |
, lname char(10) default 'Felici' | |
, bdate date | |
, comment varchar(100) | |
) | |
;</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>If you have an input file containing:</p> | |
</div> | |
<hr> | |
<div class="paragraph"> | |
<p>ignoreme,<strong>Maurizio</strong>,xyz,<em>commentM</em>, ignore,remaining, fields | |
ignoreme,<strong>Lucia</strong>,xyz,<em>commentL</em>, ignore, this | |
ignoreme,<strong>Giovanni</strong>,xyz,<em>commentG</em>, ignoreme,Antonella,xyz,commentA | |
<strong>*</strong></p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>Bold text</strong> represents <code>fname</code>.</p> | |
</li> | |
<li> | |
<p><em>Italic text</em> represents <code>comment</code>.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>and a map-file like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">FNAME:1 BDATE:CDATE COMMENT:4</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Then:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>First column (<code>ID</code>) is loaded with its default value (not in the map file)</p> | |
</li> | |
<li> | |
<p>Second column (<code>FNAME</code>) is loaded with the second input field from file (<code>FNAME:1</code>)</p> | |
</li> | |
<li> | |
<p>Third column (<code>LNAME</code>) is loaded with its default value (not in the map file)</p> | |
</li> | |
<li> | |
<p>Fourth column (<code>BDATE</code>) is loaded with the Current Data generated by odb (<code>BDATE:CDATE</code>)</p> | |
</li> | |
<li> | |
<p>Fifth column (<code>COMMENT</code>) is loaded with the fifth column in the input file (<code>COMMENT:4</code>)</p> | |
</li> | |
</ul> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="load_binary_files">5.7. Load Binary Files</h3> | |
<div class="paragraph"> | |
<p>Assuming that your back-end database (and your ODBC Driver) supports BLOB data types, or equivalent, | |
you can use odb to directly load binary (or any other) files into a database column using the <code>[:em=char]</code> symbol | |
to identify the file to be loaded into that specific database field.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>Suppose you have a table like this (MySQL):</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">create table pers.myphotos | |
( id integer | |
, image mediumblob | |
, phts timestamp | |
) | |
;</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Then, you can load a file like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ cat myphotos.csv | |
001,@/home/mauro/images/image1.jpg,2012-10-21 07:31:21 | |
002,@/home/mauro/images/image2.jpg,2012-10-21 07:31:21 | |
003,@/home/mauro/images/image3.jpg,2012-10-21 07:31:21</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>by running a command like this:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn -l src=myphotos.csv:tgt=pers.myphotos:em=\@</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>odb considers the string following the “em” character as the path of the file to be loaded in that specific field.</p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
odb does not load rows where the size of the input file is greater than the target database column. | |
</td> | |
</tr> | |
</table> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="load_xml_files">5.8. Load XML Files</h3> | |
<div class="paragraph"> | |
<p>Trafodion odb supports loading XML files into tables, the key construct for XML files can be an element or an attribute.</p> | |
</div> | |
<div class="sect3"> | |
<h4 id="_load_xml_files_where_data_is_stored_in_element_nodes">5.8.1. Load XML Files Where Data is Stored in Element Nodes</h4> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p>Create a table.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">./odb64luo -x "create table testxmlload(id int, name char(20))"</code></pre> | |
</div> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic" start="2"> | |
<li> | |
<p>Suppose you have a xml file where data is stored in element nodes like the following.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">-bash-4.1$ cat test.xml | |
<?xml version="1.0" encoding="UTF-8"?> | |
<data> | |
<id>1</id> | |
<name>hello</name> | |
</data></code></pre> | |
</div> | |
</div> | |
<div class="admonitionblock tip"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-tip" title="Tip"></i> | |
</td> | |
<td class="content"> | |
To check what will be loaded before loading XML file into table, run the following command | |
<code>./odb64luo -l src=test.xml :tgt=testxmlload:xmltag=data:xmldump</code> | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic" start="3"> | |
<li> | |
<p>Load the test.xml file into the table, run the following command.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">./odb64luo -l src=test.xml:tgt=testxmlload:xmltag=data</code></pre> | |
</div> | |
</div> | |
<div class="admonitionblock tip"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-tip" title="Tip"></i> | |
</td> | |
<td class="content"> | |
<code>xmltag=data</code> means odb will load data from the element nodes. For more information, see <a href="#_data_loading_operators">Data Loading Operators</a>. | |
</td> | |
</tr> | |
</table> | |
</div> | |
</div> | |
<div class="sect3"> | |
<h4 id="_load_xml_files_where_data_is_stored_in_attribute_nodes">5.8.2. Load XML Files Where Data is Stored in Attribute Nodes</h4> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p>Create a table.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">./odb64luo -x "create table testxmlload(id int, name char(20))"</code></pre> | |
</div> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic" start="2"> | |
<li> | |
<p>Suppose you have a XML file where data is stored in attribute nodes like the following.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">-bash-4.1$ cat test.xml | |
<?xml version="1.0" encoding="UTF-8"?> | |
<data id="1" name="hello"></data></code></pre> | |
</div> | |
</div> | |
<div class="admonitionblock tip"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-tip" title="Tip"></i> | |
</td> | |
<td class="content"> | |
To check what will be loaded before loading XML file into table, run the following command. | |
<code>./odb64luo -l src=test.xml:tgt=testxmlload:xmltag=data:xmldump</code> | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic" start="3"> | |
<li> | |
<p>Load the test.xml file into the table, run the following command.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">./odb64luo -l src=test.xml:tgt=testxmlload:xmltag=+data</code></pre> | |
</div> | |
</div> | |
<div class="admonitionblock tip"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-tip" title="Tip"></i> | |
</td> | |
<td class="content"> | |
<code>xmltag=+data</code> (with a plus sign specified) means odb will load data from the attribute nodes. For more information, see <a href="#_data_loading_operators">Data Loading Operators</a>. | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_reduce_the_odbc_buffer_size">5.9. Reduce the ODBC Buffer Size</h3> | |
<div class="paragraph"> | |
<p>odb allocates memory for the ODBC buffers during load/extract operations based on the max possible length of the | |
source/target columns.</p> | |
</div> | |
<div class="paragraph"> | |
<p>If you have a column defined as <code>VARCHAR(2000</code>), then odb allocates enough space for 2,000 characters in the ODBC buffer.</p> | |
</div> | |
<div class="paragraph"> | |
<p>If you know in advance that you never will load/extract 2,000 characters, then you can limit the amount of space allocated by odb. | |
This reduces memory usage and increase performances because of the reduced network traffic.</p> | |
</div> | |
<div class="paragraph"> | |
<p>Given the following table:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -u xxx -p xxx -d traf -i D:TRAFODION.USR.TMX | |
odb [2015-04-20 21:41:38]: starting ODBCconnection(s)... 0 | |
Connected to Trafodion | |
CREATE TABLE TRAFODION.USR."TMX" | |
( ID INTEGER NOT NULL | |
, NAME VARCHAR(400) | |
, PRIMARY KEY (ID) | |
) | |
;</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>And an input file that contains:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat tmx.dat | |
1,Maurizio | |
2,Lucia | |
3,Martina | |
4,Giovanni | |
5,Marco | |
6,Roland | |
7,Randy | |
8,Paul | |
9,Josef | |
10,Some other name</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The max length of the second field in this file is:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ awk -F\, 'BEGIN\{max=0} \{if(NF==2)\{len=length($i);if(len>max)max=len}} | |
END\{print max}' tmx.dat | |
15</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>In this case you can use <code>:maxlen=15</code> to limit the amount of the ODBC buffer:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -u xxx -p xxx -d traf -l src=tmx.dat:tgt=usr.tmx:truncate:maxlen=15 | |
odb [2015-04-20 21:46:11]:starting ODBC connection(s)... 0 | |
Connected to Trafodion | |
[0.0.0]--- 0 row(s) deleted in 0.052s (prep 0.012s, exec 0.040s, fetch 0.000s/0.000s) | |
[0] 10 records inserted [commit] | |
[0] odb version 1.3.0 Load(2) statistics: | |
[0] Target table: (null).USR.TMX | |
[0] Source: tmx.dat | |
[0] Pre-loading time: 1.254 s (00:00:01.254) | |
[0] Loading time: 0.026 s(00:00:00.026) | |
[0] Total records read: 10 | |
[0] Total records inserted: 10 | |
[0] Total number of columns: 2 | |
[0] Total bytes read: 99 | |
[0] Average input row size: 9.9 B | |
[0] ODBC row size: *26 B (data) + 16 B (len ind)* | |
[0] Rowset size: 100 | |
[0] Rowset buffer size: *4.10 KiB* | |
[0] Load throughput (real data): 3.718 KiB/s | |
[0] Load throughput (ODBC): 9.766 KiB/s | |
odb [2015-04-20 21:46:12]: exiting. Session Elapsed time 1.294 seconds (00:00:01.294)</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>If you do not specify this parameter odb allocates the buffer for the max possible length of each field:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -u xxx -p xxx -d traf -l src=tmx.dat:tgt=usr.tmx:truncate | |
odb [2015-04-20 21:47:13]: starting ODBC connection(s)... 0 | |
Connected to Trafodion | |
[0.0.0]--- 10 row(s) deleted in 0.107s (prep 0.012s, exec 0.095s, fetch 0.000s/0.000s) | |
[0] 10 records inserted [commit] | |
[0] odb version 1.3.0 Load(2) statistics: | |
[0] Target table: (null).USR.TMX | |
[0] Source: tmx.dat | |
[0] Pre-loading time: 1.330 s (00:00:01.330) | |
[0] Loading time: 0.032 s(00:00:00.032) | |
[0] Total records read: 10 | |
[0] Total records inserted: 10 | |
[0] Total number of columns: 2 | |
[0] Total bytes read: 99 | |
[0] Average input row size: 9.9 B | |
[0] ODBC row size: 411 B (data) + 16 B (len ind) | |
[0] Rowset size: 100 | |
[0] Rowset buffer size: 41.70 KiB | |
[0] Load throughput (real data): 3.021 KiB/s | |
[0] Load throughput (ODBC): 125.427 KiB/s | |
odb [2015-04-20 21:47:14]: exiting. Session Elapsed time 1.373 seconds (00:00:01.373)</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_extract_tables">5.10. Extract Tables</h3> | |
<div class="paragraph"> | |
<p>You can use odb to extract tables from a database and write them to standard files (or named pipes).</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn –T 3 \ | |
-e src=TRAFODION.MAURIZIO.LIN%:tgt=$\{DATA}/ext_%t.csv.gz:rows=m10:fs=\|:trim:gzip: \ | |
-e src=TRAFODION.MAURIZIO.REGION:tgt=$\{DATA}/ext_%t.csv.gz:rows=m10:fs=\|:trim:gzip \ | |
-e src=TRAFODION.MAURIZIO.NATION:tgt=$\{DATA}/ext_%t.csv.gz:rows=m10:fs=\|:trim:gzip</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The example above:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Extracts tables <code>REGION</code>, <code>NATION</code>, and all tables starting with <code>LIN</code> from <code>TRAFODION.MAURIZIO</code> schema.</p> | |
</li> | |
<li> | |
<p>Saves data into files <code>ext_%t.csv.gz</code> (<code>%t</code> is expanded to the real table name).</p> | |
</li> | |
<li> | |
<p>Compresses the output file (gzip) on the fly (uncompressed data never lands to disk).</p> | |
</li> | |
<li> | |
<p>Trims text fields.</p> | |
</li> | |
<li> | |
<p>Uses a 10 MB IO buffer.</p> | |
</li> | |
<li> | |
<p>Uses three threads (ODBC connection) for the extraction process.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="sect3"> | |
<h4 id="_extraction_options">5.10.1. Extraction Options</h4> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">-e {src={table|-file}|sql=<customsql>}:tgt=[+]file[:pwhere=where_cond] | |
[:fs=fieldsep][:rs=recsep][:sq=stringqualifier][:ec=escape_char][:soe] | |
[:ns=nullstring][es=emptystring][:rows=#rowset][:nomark][:binary][:fwc] | |
[:max=#max_rec][:trim=[cCvVdt]][:rtrim][:cast][:multi][:efs=string] | |
[:parallel=number][:gzip][:gzpar=wb??][:uncommitted][:splitby=column] | |
[:pre={@sqlfile}|{[sqlcmd]}[:mpre=\{@sqlfile}|{[sqlcmd]}[:post={@sqlfile}|{[sqlcmd]}] | |
[tpar=#tables][:time][:nts][:cols=[-]columns]][:maxlen=#bytes][:xml]</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>The following table describes each extract operator:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 30%;"> | |
<col style="width: 70%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Extract option</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>src=<CAT.SCH.TAB>|-file</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines the source table(s). You can use:<br> | |
<br> | |
- a single table name (for example TRAFODION.MFTEST.LINEITEM)<br> | |
- a group of tables (for example TRAFODION.MFTEST.LIN%)<br> | |
- a file containing a list of tables to extract (<code>-</code> should precede the filename)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>sql=<sql></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">A custom SQL command you can use to extract data. This is <strong>alternative</strong> to <code>src=</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tgt=[+]file</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Output file. You can use the following keywords for this field:<br> | |
<br> | |
- <code>%t/%T</code> expands to the (lower/upper case) table name<br> | |
- <code>%s/%S</code> expands to the (lower/upper case) schema name<br> | |
- <code>%c/%C</code> expands to the (lower/upper case) catalog name<br> | |
- <code>%d</code> expands to the extraction date (YYYYMMDD format)<br> | |
- <code>%D</code> expands to the extraction date (YYYY-MM-DD format)<br> | |
- <code>%m</code> expands to the extraction time (hhmmss format)<br> | |
- <code>%M</code> expands to the extraction time (hh:mm:ss format)<br> | |
- <code>stdout</code> prints the extracted records to the standard output.<br> | |
<br> | |
If you add a <code>+</code> sign in front of the file-name, then odb <strong>appends</strong> to <code>file</code> instead of <strong>creates</strong> <code>file</code>.<br> | |
<br> | |
<code>hdfs./<hdfspath>/<file></code> to write exported table under the Hadoop File Distributed System (HDFS).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>fs=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Field separator. You can define the field separator as:<br> | |
<br> | |
- a normal character (for example <code>fs=,</code>)<br> | |
- ASCII decimal (for example <code>fs=44</code> - 44 means comma)<br> | |
- ASCII octal value (for example <code>fs=054</code> – 054 means comma)<br> | |
- ASCII hex value (for example <code>fs=x2C</code> – x2C means comma)<br> | |
<br> | |
The default field separator is <code>,</code> (comma)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rs=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Record separator. You can define the record separator the same way as the field separator.<br> | |
<br> | |
The default record separator is <code>\n</code> (new line)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>max=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Max number of records to extract.<br> | |
<br> | |
The default is to extract all records</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>sq=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The string qualifier character used to enclose strings. You can | |
define the string qualifier the same way as the field separator</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ec=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Character used as escape character. You can define the | |
escape character the same way as the field separator.<br> | |
<br> | |
Default is <code>\</code> (back slash).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rows=<num>|k<num>|m<num></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines the size of the I/O buffer for each extraction thread. You | |
can define the size of this buffer in two different ways:<br> | |
<br> | |
- number of rows (for example: <code>rows=100</code> means 100 rows as IO buffer)<br> | |
- buffer size in kB or MB (for example: <code>rows=k512</code> (512 kB buffer) or <code>rows=m20</code> (20MB buffer))</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ns=<nullstring></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">How odb represents NULL values in the output file.<br> | |
<br> | |
Default is the empty string (two field separators one after the other)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>es=<emptystring></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">How odb represents VARCHAR empty strings (NOT NULL with zero | |
length) values in the output file.<br> | |
<br> | |
Default is the empty string (two field separators one after the other)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>gzpar=<params></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This are extra parameters you can pass to <em>tune</em> the gzip compression algorithm.<br> | |
<br> | |
<strong>Examples</strong><br> | |
<br> | |
- <code>gzpar=wb9</code>: max compression (slower)<br> | |
- <code>gzpar=wb1</code>: basic compression (faster)<br> | |
- <code>gzpar=wb6h</code>: Huffman compression only<br> | |
- <code>gzpar=wb6R</code>: Run-length encoding only</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>trim[=<params>]</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Accept the following optional parameters:<br> | |
- <code>c</code> trims<sup>1</sup> from CHAR<sup>2</sup>.<br> | |
- <code>C</code> trims trailing spaces from CHAR<sup>2</sup><br> | |
- <code>v</code> trims leading spaces from VARCHAR fields<br> | |
- <code>V</code> trims trailing spaces from VARCHAR fields<br> | |
- <code>d</code> trims trailing zeros after decimal sign. Example: <code>12.3000</code> is extracted as <code>12.3</code>.<br> | |
- <code>t</code> trims decimal portion from TIME/TIMESTAMP fields. For example: <code>1999-12-19 12:00:21.345</code> is extracted as <code>1999-12-19 12:00:21</code>.<br> | |
<br> | |
<strong>Trim Examples</strong><br> | |
<br> | |
<code>:trim=cC</code> —> <em>trims leading/trailing spaces from CHAR fields</em>.<br> | |
<code>:trim=cCd</code> —> <em>trims leading/trailing spaces from CHARs and trailing decimal zeroes</em>.<br> | |
<br> | |
If you do not specify any argument for this operator odb uses <code>cCvV</code>. In other words <code>:trim:</code> is a shortcut for <code>:trim=cCvV:</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nomark</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Don’t print the number of records extracted so far by each thread.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>soe</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Stop On Error. odb stop as soon as it encounters an error.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>parallel=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb uses as many threads as the parallel argument to extract data from partitioned source tables. <strong>You have to use splitby.</strong><br> | |
<br> | |
Each thread takes care of a specific range of the source table partitions. For example if you specify <code>parallel=4</code> and the source table | |
is made of 32 partitions, then odb starts <strong>four</strong> threads (four ODBC connections):<br> | |
<br> | |
- thread 0 extracts partitions 0-7<br> | |
- thread 1 extracts partitions 8-15<br> | |
- thread 2 extracts partitions 16-23<br> | |
- thread 3 extracts partitions 24-31</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>multi</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This option can be used in conjunction with parallel operator to write as many output files as the number of extraction | |
threads. Output file names are built adding four digits at the end of the file identified by the <code>tgt</code> operator.<br> | |
<br> | |
For example, with <code>src=trafodion.mauro.orders:tgt=%t.csv:parallel=4:multi</code><br> | |
<br> | |
odb writes into the following output files:<br> | |
<br> | |
- <code>orders.csv.0001</code><br> | |
- <code>orders.csv.0002</code><br> | |
- <code>orders.csv.0003</code><br> | |
- <code>orders.csv.0004</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pwhere=<where condition></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This option is used in conjunction with parallel limiting the | |
extraction to records satisfying the where condition.<br> | |
<br> | |
NOTE: The where condition is limited to columns in the source table.<br> | |
<br> | |
For example: you want to extract records with <code>TRANS_TS > 1999-12-12 09:00:00</code> from the source table TRAFODION.MAURO.MFORDERS | |
using eight parallel streams to a single, gzipped, file having the same name as the source table:<br> | |
<br> | |
<code>src=trafodion.mauro.mforders:tgt=%t.gz:gzip:parallel=8:pwhere=[TRANS_TS > TIMESTAMP ‘1999-12-12 09:00:00’]…</code><br> | |
<br> | |
You can enclose the where condition between square brackets to avoid a misinterpretation of the characters in the where condition.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>errmax=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb prints up to num error messages per rowset. Normally used with soe to limit the number of error messages printed to the standard error stream.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>uncommitted</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Adds FOR READ UNCOMMITTED ACCESS to the select(s) command(s).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rtrim</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">RTRIM() CHAR columns on the server. From a functional point of view this is equivalent to <code>trim=C</code> but <code>rtrim</code> is executed on the server so | |
it saves both client CPU cycles and network bandwidth.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>cast</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Performs a (server side) cast to VARCHAR for all non-text columns. Main scope of this operator is to <em>move</em> CPU cycles from the client to | |
the database server. It increases network traffic. To be used when:<br> | |
<br> | |
- the extraction process is CPU bound on the client AND<br> | |
- network has a lot of available bandwidth AND<br> | |
- database server CPUs are not <em>under pressure</em>.<br> | |
<br> | |
Tests extracting a table full of NUMERIC(18,4), INT and DATES shows:<br> | |
<br> | |
- client CPU cycles down ~50% on the client<br> | |
- network traffic up ~40%</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>splitby=<column></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This operator let you to use parallel extract from any database. <strong><column> has to be a SINGLE, numeric column</strong>. | |
odb calculates min()/max() value for <code><column></code> and assign to each <parallel> thread the extraction of the rows in its <em>bucket</em>.<br> | |
<br> | |
For example, if you have:<br> | |
<br> | |
<code>…:splitby=emp_id:parallel=4…</code><br> | |
<br> | |
with <code>min(emp_id)=1</code> and <code>max(emp_id)=1000</code>, the four threads extract the following rows:<br> | |
<br> | |
<code>thread #0 emp_id >=1 and emp_id < 251</code><br> | |
<code>thread #1 emp_id >=251 and emp_id < 501</code><br> | |
<code>thread #2 emp_id >=501 and emp_id < 751</code><br> | |
<code>thread #3 emp_id >=751 and emp_id < 1001</code> (odb uses max(emp_id) + 1)<br> | |
<br> | |
If the values are not equally distributed, then data extraction is de-skewed.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pre={@sqlfile}|{[sqlcmd]</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb runs a <strong>single instance</strong> of either the <code>sqlfile</code> script or <code>sqlcmd</code> SQL | |
command (enclosed between square brackets) on the <strong>source system</strong> immediately before table extraction.<br> | |
<br> | |
Source table won’t be extracted if SQL execution fails and Stop On Error is set.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>mpre={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Each odb thread runs either sqlfile script or sqlcmd SQL command (enclosed between | |
square brackets)on the <strong>source system</strong> immediately before table extraction. You can use <code>mpre</code> to set database specific | |
features <strong>for each extraction thread</strong>.<br> | |
<br> | |
<strong>Examples</strong><br> | |
<br> | |
1. You want <strong>Trafodion</strong> to ignore missing stats warning. Then you can run via <code>mpre</code> a SQL script containing:<br> | |
<br> | |
<code>control query default HIST_MISSING_STATS_WARNING_LEVEL '0';</code><br> | |
<br> | |
2. You want <strong>Oracle</strong> to extract dates in the <code>YYYY-MM-DD hh:mm:ss</code> format. Then you can run via <code>mpre</code> a script containing:<br> | |
<br> | |
<code>ALTER SESSION SET NLS_DATE_FORMAT='YYYY-MM-DD HH:MI:SS'</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>post={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb runs a <strong>single instance</strong> of either a <code>sqlfile</code> script or <code>sqlcmd</code> SQL | |
command (enclosed between square brackets) on the <strong>source system</strong> immediately after table extraction.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpar=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb extracts <code>num</code> tables in parallel when <code>src</code> is a list of files to be loaded.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>maxlen=#bytes</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb limits the amount of memory allocated in the ODBC buffers for CHAR/VARCHAR fields to <code>#bytes</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>xml</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Writes output file in XML format</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>time</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb prints a <em>timeline</em> (milliseconds from start).</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p>The following characters are considered <em>spaces</em>: <code>blank</code>, <code>tab</code>, <code>new line</code>, <code>carriage return</code>, <code>form feed</code>, and <code>vertical tab</code>.</p> | |
</li> | |
<li> | |
<p>When the source table column is defined as NOT NULL and the specific field contains only blanks, odb leaves in the output file | |
one single blank. This helps to distinguish between NULL fields (<code><field_sep><field_sep></code>) and NOT NULL fields containing all blanks | |
(<code><field_sep><blank><field_sep></code>).</p> | |
</li> | |
</ol> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_extract_a_list_of_tables">5.11. Extract a List of Tables</h3> | |
<div class="paragraph"> | |
<p>You can use odb to extract all tables listed in a file.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat tlist.txt | |
# List of tables to extract src=TRAFODION.MAURIZIO.ORDERS | |
src=TRAFODION.MAURIZIO.CUSTOMER src=TRAFODION.MAURIZIO.PART | |
src=TRAFODION.MAURIZIO.LINEITEM</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You can extract all these tables by running:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn -e src=-tlist.txt:tgt=%t_%d%m:rows=m20:sq=\”</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Please note the <code>src=-tlist.txt</code>.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_copy_tables_from_one_database_to_another">5.12. Copy Tables From One Database to Another</h3> | |
<div class="paragraph"> | |
<p>odb can directly copy tables from one data-source to another. For example, from Trafodion to Teradata or vice-versa). | |
Data <strong><em>never lands to disk</em></strong> when using this option.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The target table has to be be created in advance and should have a compatible structure.</p> | |
</div> | |
<div class="sect3"> | |
<h4 id="_copy_operators">5.12.1. Copy Operators</h4> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">-cp src={table|-file:tgt=schema[.table][pwhere=where_cond][:soe][:nts] | |
[:truncate][:rows=#rowset][:nomark][:max=#max_rec][:fwc][:bpwc=#] | |
[:parallel=number][errmax=#max_err][:commit=auto|end|#rows|x#rs][:time] | |
[:direct][:uncommitted][:norb][:splitby=column][:pre={@sqlfile}|{[sqlcmd]}] | |
[:post={@sqlfile}|{[sqlcmd]}][:mpre={@sqlfile}|{[sqlcmd]}][:ifempty] | |
[:loaders=#loaders][:tpar=#tables][:cols=[-]columns] | |
[sql={[sqlcmd]|@sqlfile|-file}[:bind=auto|char|cdef] | |
[tmpre={@sqlfile}|{[sqlcmd]}][seq=field#[,start]]</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Complete list of the Copy Operators:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 33.3333%;"> | |
<col style="width: 66.6667%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Copy Operator</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>src=<CAT.SCH.TAB>|-file</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Defines the source table(s). You can use:<br> | |
<br> | |
- a single table (for example: TRAFODION.MFTEST.LINEITEM)<br> | |
- a group of tables (for example: TRAFODION.MFTEST.LIN%)<br> | |
- a file containing a list of tables to copy (‘-‘ should precede the filename)</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tgt=<CAT.SCH.TAB></code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Target table(s). You can use the following keywords for this field:<br> | |
<br> | |
- <code>%t/%T</code>: Expands to the (lower/upper case) source table name.<br> | |
- <code>%s/%S</code>: Expands to the (lower/upper case) source schema name.<br> | |
- <code>%c/%C</code>: Expands to the (lower/upper case) source catalog name.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>sql={[sqlcmd]|@sqlfile|-file}</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb uses a generic SQL — instead of a <em>real</em> table — as source.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>max=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>This is the max number of records to copy. Default is to copy all records in the source table.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rows=<num>|k<num>|m<num></code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Defines the size of the I/O buffer for each copy thread. You can | |
define the size of this buffer in two different ways:<br> | |
<br> | |
- number of rows (for example: <code>rows=100</code> means 100 rows as IO buffer)<br> | |
- buffer size in kB or MB (for example: <code>rows=k512</code> (512 kB buffer) or <code>rows=m20</code> (20MB buffer))</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>truncate</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Truncates the target table before loading.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ifempty</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Loads the target table only if empty.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nomark</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Don’t print the number of records loaded so far during loads.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>soe</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Stop On Error. odb stops as soon as it encounters an error.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>parallel=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb uses two kinds of threads:<br></p> | |
</div> | |
<div class="paragraph"> | |
<p>• To extract data from partitioned source table.<br> | |
The number of the threads is as many as the parallel argument.<br></p> | |
</div> | |
<div class="paragraph"> | |
<p>• To write data to the target table.<br> | |
The number of the threads is equal to <strong>parallel argument</strong> * <strong>number of loaders</strong>.<br></p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>If you specify parallel argument = 4 and the source table has 32 partitions, then odb start:<br></p> | |
</div> | |
<div class="paragraph"> | |
<p>• 4 threads (4 ODBC connections) to read from the source table.</p> | |
</div> | |
<div class="paragraph"> | |
<p>• 8 threads (8 ODBC connections = 4 [parallel argument] * 2 [default number of loaders]) to write to the target table.</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 33.3333%;"> | |
<col style="width: 66.6667%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Thread</th> | |
<th class="tableblock halign-left valign-top">Task</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 0</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Extracts partitions 0-7 from source.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 1 and Thread 2</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Write data extracted from thread 0 to target.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 3</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Extracts partitions 8-15 from source.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 4 and Thread 5</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Write data extracted from thread 3 to target.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 6</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Extracts partitions 16-23 from source.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 7 and Thread 8</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Write data extracted from thread 6 to target.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 9</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Extracts partitions 24-31 from source.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread 10 and Thread 11</p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Write data extracted from thread 9 to target.</p> | |
</div></div></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p><strong>You have to specify splitby.</strong></p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pwhere=<where condition></code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Used in conjunction with parallel to copy only records satisfying the where condition.<br> | |
<br> | |
<strong>Note:</strong> The where condition is limited to columns in the source table.<br> | |
<br> | |
<strong>Example</strong><br> | |
<br> | |
You want to copy records with <code>TRANS_TS > 1999-12-12 09:00:00</code> from the source table TRAFODION.MAURO.MFORDERS using eight parallel | |
streams to a target table having the same name as the source table:<br> | |
<br> | |
<code>src=trafodion.mauro.mforders:tgt=trafodion.dest_schema.%t:parallel=8:pwhere=[TRANS_TS > TIMESTAMP ‘1999-12-12 09:00:00’]…</code><br> | |
<br> | |
You can enclose the where condition between square brackets to avoid a misinterpretation of the characters in the where condition.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>commit=auto|end|#rows|x#rs</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Defines how odb will commit the inserts. You have the following choices:<br> | |
<br> | |
- <code>auto</code> (Default) &8212; Commits every single insert (see also rows load operator). <code>end</code> commits when all rows (assigned to a given thread) have been inserted.<br> | |
- <code>#rows</code> — Commits every <code>#rows</code> copied rows.<br> | |
- <code>x#rs</code> — Commits every <code>#rs</code> rowset copied. (See <code>:rows</code>)</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>direct</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Adds <code>/*+ DIRECT */</code> hint to the insert statement. To be used with Vertica databases in order to store | |
inserted rows <em>directly</em> into the Read-Only Storage (ROS). See Vertica’s documentation.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>errmax=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb prints up to num error messages per rowset. Normally used with soe to limit the number of | |
error messages printed to the standard error stream.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>uncommitted</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Adds <code>FOR READ UNCOMMITTED ACCESS</code> to the <code>select(s) command(s)</code>.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>splitby=<column></code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Lets you to use parallel copy from any database. | |
<strong><column> has to be a SINGLE, numeric column</strong>. odb calculates min()/max() value for <code><column></code> and assigns to each | |
<code><parallel></code> thread the extraction of the rows in its <em>bucket</em>.<br> | |
<br> | |
For example, if you have:<br> | |
<br> | |
<code>…:splitby=emp_id:parallel=4…</code><br> | |
<br> | |
with <code>min(emp_id)=1</code> and <code>max(emp_id)=1000</code>, then the four threads extracts the following rows:<br> | |
<br> | |
<code>thread #0 emp_id >=1 and emp_id < 251</code><br> | |
<code>thread #1 emp_id >=251 and emp_id < 501</code><br> | |
<code>thread #2 emp_id >=501 and emp_id < 751</code><br> | |
<code>thread #3 emp_id >=751 and emp_id < 1001 (odb uses max(emp_id) + 1)</code><br> | |
<br> | |
If the values are not equally distributed data extraction is de-skewed.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pre={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb runs a <strong>single instance</strong> of either a <code>sqlfile</code> script or <code>sqlcmd</code> | |
(enclosed between square brackets) on the <strong>target system</strong> immediately before loading the target table. | |
You can, for example, CREATE the target table before loading it.<br> | |
<br> | |
The target table isn’t loaded if SQL execution fails and Stop On Error is set.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>mpre={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Each odb thread runs either a <code>sqlfile</code> script or <code>sqlcmd</code> | |
(enclosed between square brackets) on the <strong>source system</strong> immediately before | |
loading the target table. You can use <code>mpre</code> to set database specific features for each thread.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tmpre={@sqlfile}|{[sqlcm d]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Each odb thread runs either a <code>sqlfile</code> script or <code>sqlcmd</code> | |
(enclosed between square brackets) on the <strong>target system</strong> immediately before loading the target table. | |
You can use <code>mpre</code> to set database specific features for each thread.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>post={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb runs a <strong>single instance</strong> of either a <code>sqlfile</code> script or <code>sqlcmd</code> | |
(enclosed between square brackets) on the <strong>target system</strong> immediately after the target table has been | |
loaded. You can, for example, update database stats after loading a table.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpar=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb copies <code>num</code> tables in parallel when <code>src</code> is a list of files to be loaded.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>loaders=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb uses <code>num</code> load threads for each extract thread. Default is two loaders per extractor,</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>fwc</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>Force Wide Characters. odb considers SQL_CHAR/SQL_VARCHAR fields as they were defined SQL_WCHAR/SQL_WVARCHAR.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>bpwc=#</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb internally allocates 4 bytes/char for SQL_WCHAR/SQL_WVARCHAR columns. | |
You can modify the number of bytes allocated for each char using this parameter.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>bind=auto|char|cdef</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb can bind columns to ODBC buffer as characters (char) or <code>C Default</code> data types (<code>cdef</code>). | |
The default (<code>auto</code>) uses <code>cdef</code> if <code>SRC/TGT</code> use the same database or char if <code>SRC/TGT</code> databases differ.</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>seq=field#[,start]</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb adds a sequence when loading the target system on column number <code>field#</code>. | |
You can optionally define the sequence start value. (Default: 1)</p> | |
</div></div></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>time</code></p></td> | |
<td class="tableblock halign-left valign-top"><div><div class="paragraph"> | |
<p>odb prints a <em>timeline</em> (milliseconds from start).</p> | |
</div></div></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>When copying data from one data source to another, odb needs user/password/dsn for both source and target system. | |
User credentials and DSN for the target system are specified this way:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u src_user:tgt_user –p src_pwd:tgt:pwd –d src_dsn:tgt_dsn ... -cp src=...:tgt=...</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_copy_a_list_of_tables">5.13. Copy a List of Tables</h3> | |
<div class="paragraph"> | |
<p>You can use odb to copy a list of tables from one database to another.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat tlist.txt | |
# List of tables to extract | |
src=TRAFODION.MAURIZIO.ORDERS | |
src=TRAFODION.MAURIZIO.CUSTOMER | |
src=TRAFODION.MAURIZIO.PART | |
src=TRAFODION.MAURIZIO.LINEITEM</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You can extract all these tables by running:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user1:user2 –p xx:yy –d dsn1:dsn2 \ | |
-cp src=-tlist.txt:tgt=tpch.stg_%t:rows=m2:truncate:parallel=4 -T 8</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Please note the <code>src=-tlist.txt</code>. This command copies:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 50%;"> | |
<col style="width: 50%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Source</th> | |
<th class="tableblock halign-left valign-top">Target</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TRAFODION.MAURIZIO.ORDERS</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpch.stg_orders</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TRAFODION.MAURIZIO.CUSTOMER</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpch.stg_customer</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TRAFODION.MAURIZIO.PART</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpch.stg_part</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TRAFODION.MAURIZIO.LINEITEM</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpch.stg_lineitem</code></p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>Optionally, you can define any other <em>command line</em> options in the input | |
file.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>Using different <em>splitby columns</em>.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat tlist2.txt | |
# List of tables to extract and their “splitby columns” | |
src=TRAFODION.MAURIZIO.ORDERS:splitby=O_ORDERKEY | |
src=TRAFODION.MAURIZIO.CUSTOMER:splitby=C_CUSTOMERKEY | |
src=TRAFODION.MAURIZIO.PART:splitby=P_PARTKEY | |
src=TRAFODION.MAURIZIO.LINEITEM:splitby=L_PARTKEY</code></pre> | |
</div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_case_sensitive_table_and_column_names">5.14. Case-Sensitive Table and Column Names</h3> | |
<div class="paragraph"> | |
<p>Your database configuration determines whether you can use case sensitive table/column names. | |
odb maintains table/column case sensitiveness when they are enclosed in double quotes.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>The following commands create a <code>TRAFODION.MAURIZIO.Names</code> table made of three columns: | |
“name”, “NAME” and “Name”.</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">create table trafodion.maurizio."Names" | |
( "name" char(10) | |
, "NAME" char(10) | |
, "Name" char(10) | |
) | |
no partitions;</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Double quotes have to be escaped under *nix. A few examples:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -i T:trafodion.maurizio.\"Names\" | |
~/Devel/odb $ ./odb64luo -x "select from trafodion.maurizio.\"Names\"" | |
~/Devel/odb $ ./odb64luo -l src=names.txt:tgt=trafodion.maurizio. | |
\"Names\":map=names.map:pc=32</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>You can omit double quotes around column names when using <em>mapfiles</em>.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_determine_appropriate_number_of_threads_for_load_extract_copy_diff">5.15. Determine Appropriate Number of Threads for Load/Extract/Copy/Diff</h3> | |
<div class="paragraph"> | |
<p>If you have to load/extract or copy multiple tables in parallel the best option is to use the options | |
<code>:tpar=number</code> and <code>:parallel=number</code>. <code>:tpar</code> defines how many tables have to be copied/extracted | |
in parallel; <code>:parallel</code> defines how many <em>data streams</em> to use for each table. This way, odb automatically | |
allocates and start the “right” number of threads.</p> | |
</div> | |
<div class="paragraph"> | |
<p>A rule of thumb when copying/loading or extracting tables is to use as many <em>data streams</em> as: | |
<code>min(number of middle-tier CPUs, number of source CPUs, number of target CPUs)</code></p> | |
</div> | |
<div class="paragraph"> | |
<p>The number of threads started for each <em>data stream</em> depend on the operation type:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 15%;"> | |
<col style="width: 30%;"> | |
<col style="width: 40%;"> | |
<col style="width: 15%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Operation</th> | |
<th class="tableblock halign-left valign-top">Total threads</th> | |
<th class="tableblock halign-left valign-top">Explanation</th> | |
<th class="tableblock halign-left valign-top">Example with <code>parallel=4</code></th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Load</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">parallel + 1</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">One thread to read from file + one thread per parallel to load.</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">5</p></td> | |
</tr> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Extract</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">parallel</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">One thread per parallel to extract.</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">4</p></td> | |
</tr> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Copy</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">parallel * (1+loaders)</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Two threads per parallel: read from source and write to target.</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">12 (if loaders=2)</p></td> | |
</tr> | |
<tr> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Diff</p></th> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">parallel * 3</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Three threads per parallel: read from source, read from target, compare.</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">12</p></td> | |
</tr> | |
</tbody> | |
</table> | |
</div> | |
<div class="sect2"> | |
<h3 id="_integrating_with_hadoop">5.16. Integrating With Hadoop</h3> | |
<div class="paragraph"> | |
<p>There are basically two ways to integrate a generic database with Hadoop using odb:</p> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p><strong>Use HIVE (Hadoop DWH) and its ODBC Driver</strong>: odb can access HIVE like any other <em>normal</em> | |
relational database. For example, you can copy to from HIVE and other databases using odb’s copy option.</p> | |
</li> | |
<li> | |
<p><strong>Add the <code>hdfs.</strong></code> prefix to the input or output file during loads/extracts*: The file is read/written | |
from/to Hadoop. odb interacts directly with the HDFS file system using <strong>libhdfs</strong>.</p> | |
<div class="paragraph"> | |
<p>This option is currently available only under Linux.</p> | |
</div> | |
</li> | |
</ol> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_comparing_tables_technology_preview">6. Comparing Tables (Technology Preview)</h2> | |
<div class="sectionbody"> | |
<div class="paragraph"> | |
<p>You can use odb to compare two tables <strong>with the same structure</strong> on different databases. | |
odb does the following to compare two tables:</p> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p>Extracts source/target tables ordered by Primary Key or any other set of columns.</p> | |
</li> | |
<li> | |
<p>Compare source/target ODBC buffers without <em>unpacking</em> them into columns/rows.</p> | |
</li> | |
</ol> | |
</div> | |
<div class="paragraph"> | |
<p>Each <em>comparison stream</em> is made up of three threads:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>One thread reading from the source table.</p> | |
</li> | |
<li> | |
<p>One thread reading from the target table.</p> | |
</li> | |
<li> | |
<p>One thread comparing the source/target buffers.</p> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p>These three threads work in parallel: the <em>compare</em> thread checks <code>buffer N</code> while the other two threads | |
extract the net block of data from the source/target database in parallel.</p> | |
</div> | |
<div class="paragraph"> | |
<p>You can have multiple <em>triplets</em> working in parallel on different section of the table using the <code>splitby</code> operator.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo -u mailto:maurizio.felici@hp.com[MFELICI:maurizio.felici@hp.com] \ | |
-d MFELICI:VMFELICI \ | |
-p xx:yy –diff src=trafodion.maurizio.lineitem:tgt=mftest.lineitem:&#42; \ | |
key=l_orderkey,l_linenumber:output=lineitem.diff: | |
rows=m2:print=IDC:&#42;splitby=l_orderkey&#42;:parallel=8</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The command above compares two tables using eight streams (<code>parallel=8</code>) made of three threads each.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The comparison threads use double buffering and advanced memory-comparison techniques. odb can provide the following information in | |
output as a CSV file:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Missing rows on target (<code>D</code> – deleted – rows) based on the <strong>key</strong> columns.</p> | |
</li> | |
<li> | |
<p>New rows on target (<code>I</code> – inserted – rows) based on the <strong>key</strong> columns.</p> | |
</li> | |
<li> | |
<p>Changed rows (same <strong>key</strong> columns but with different values in other fields).</p> | |
<div class="paragraph"> | |
<p>For these rows odb can print the original source version (<code>C</code> rows) and/or the modified target version (<code>U</code> rows).</p> | |
</div> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>odb output when comparing two tables:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ cat lineitem.diff | |
DTYPE,L_ORDERKEY,L_LINENUMBER,L_SUPPKEY,L_PARTKEY,L_QUANTITY,L_EXTENDEDPRICE,L_DISCOUNT,L_TAX,L_RETURNFLAG, | |
L_LINESTATUS,L_SHIPDATE,L_COMMITDATE,L_RECEIPTDATE,L_SHIPINSTRUCT,L_SHIPMODE,L_COMMENT | |
D,4532896,1,5974,100953,42.00,82065.90,0.03,0.00,R,F,1994-12-15,1995-01-17,1995-01-07,COLLECT COD,TRUCK,leep across the ca | |
D,4532896,2,2327,102326,48.00,63759.36,0.07,0.05,A,F,1995-02-18,1994-12-10,1995-03-12,TAKE BACK RETURN,RAIL,usly regular platelets. careful | |
D,4532896,3,612,193054,12.00,13764.60,0.05,0.02,R,F,1994-11-17,1994-11-23,1994-12-06,COLLECT COD,SHIP,s haggle quickly. ideas after the | |
D,4532896,4,9867,47362,36.00,47136.96,0.10,0.06,A,F,1995-01-05,1994-11-29,1995-01-06,COLLECT COD,RAIL,s haggle carefully bo | |
D,4532896,5,9576,2075,19.00,18564.33,0.00,0.05,R,F,1994-11-26,1995-01-17,1994-12-03,COLLECT COD,TRUCK,en sauternes integrate blithely alon | |
D,4532896,6,1016,68509,9.00,13297.50,0.07,0.00,R,F,1995-02-16,1995-01-05,1995-02-24,TAKE BACK RETURN,RAIL,ily above the blithel | |
C,1652227,3,2298,87281,28.00,35511.84,0.06,0.05,R,F,1993-05-04,1993-03-12,1993-05-12,TAKE BACK RETURN,MAIL,lly final acco | |
U,1652227,3,2298,87281,99.99,35511.84,0.06,0.05,R,F,1993-05-04,1993-03-12,1993-05-12,TAKE BACK RETURN,MAIL,lly final acco | |
D,3456226,1,8161,148160,22.00,26579.52,0.06,0.02,A,F,1994-06-26,1994-06-08,1994-07-10,DELIVER IN PERSON,FOB,uriously. furio | |
D,3456226,2,6293,108762,20.00,35415.20,0.10,0.05,R,F,1994-05-07,1994-06-03,1994-05-15,NONE,RAIL,ously bold requests along the b</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">D,3456226,3,4542,159511,33.00,51826.83,0.05,0.03,A,F,1994-07-04,1994-05-15,1994-07-26,NONE,FOB,wake carefully al | |
D,3456226,4,154,95135,33.00,37294.29,0.04,0.08,A,F,1994-05-27,1994-05-10,1994-06-14,DELIVER IN PERSON,AIR,ests. unusual dependencies wake fluffily | |
D,3456226,5,9027,126514,31.00,47755.81,0.08,0.01,R,F,1994-06-13,1994-06-18,1994-07-10,TAKE BACK RETURN,FOB,according to the arefully regular instruct | |
D,3456226,6,8477,110943,14.00,27355.16,0.03,0.01,R,F,1994-07-03,1994-05-28,1994-07-13,TAKE BACK RETURN,FOB,onic accounts. ironic,pend | |
D,3456226,7,1773,4272,34.00,39993.18,0.08,0.00,A,F,1994-05-01,1994-05-29,1994-05-15,TAKE BACK RETURN,MAIL,ounts are finally ca | |
D,3456227,7,3722,101211,22.00,26668.62,0.02,0.01,N,O,1997-12-16,1998-02-05,1997-12-19,NONE,TRUCK,uriously even platelets are fu | |
I,3456227,8,3722,101211,22.00,26668.62,0.02,0.01,N,O,1997-12-16,1998-02-05,1997-12-19,NONE,TRUCK,uriously even platelets are fu | |
I,9999999,1,8161,148160,22.00,26579.52,0.06,0.02,A,F,1994-06-26,1994-06-08,1994-07-10,DELIVER IN PERSON,FOB,uriously. furio | |
I,9999999,2,6293,108762,20.00,35415.20,0.10,0.05,R,F,1994-05-07,1994-06-03,1994-05-15,NONE,RAIL,ously bold requests along the b | |
I,9999999,3,4542,159511,33.00,51826.83,0.05,0.03,A,F,1994-07-04,1994-05-15,1994-07-26,NONE,FOB,wakecarefully al | |
I,9999999,4,154,95135,33.00,37294.29,0.04,0.08,A,F,1994-05-27,1994-05-10,1994-06-14,DELIVER IN PERSON,AIR,ests. unusual dependencies wake fluffily | |
I,9999999,5,9027,126514,31.00,47755.81,0.08,0.01,R,F,1994-06-13,1994-06-18,1994-07-10,TAKE BACK RETURN,FOB,according to the carefully regular instruct | |
I,9999999,6,8477,110943,14.00,27355.16,0.03,0.01,R,F,1994-07-03,1994-05-28,1994-07-13,TAKE BACK RETURN,FOB,onic accounts. ironic, pend | |
I,9999999,7,1773,4272,34.00,39993.18,0.08,0.00,A,F,1994-05-01,1994-05-29,1994-05-15,TAKE BACK RETURN,MAIL,ounts are finally ca</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>As you can see the first column defines the type of difference.</p> | |
</div> | |
<div class="sect2"> | |
<h3 id="_diff_operators">6.1. Diff Operators</h3> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">-diff | |
src={table|-file}:tgt=table:[key=columns][:output=[+]file][:pwhere=where_cond] | |
[:pwhere=where_cond][:nomark][:rows=#rowset][:odad][:fs=fieldsep][:time] | |
[:rs=recsep][:quick][:splitby=column][:parallel=number][:max=#max_rec] | |
[:print=[I][D][C]][:ns=nullstring][:es=emptystring][:fwc][:uncommitted] | |
[:pre={@sqlfile}|{[sqlcmd]}][:post={@sqlfile}|{[sqlcmd]}][tpar=#tables]</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Detailed Descriptions of the Copy Operators:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 30%;"> | |
<col style="width: 70%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Diff Operator</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>src=<CAT.SCH.TAB>|-file</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines the source table(s). You can use:<br> | |
<br> | |
- A single table (for example: TRAFODION.MFTEST.LINEITEM)<br> | |
- A file containing a list of tables to compare (<code>-</code> should precede the filename)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tgt=<CAT.SCH.TAB></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Target table.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>key=column[,column,…]</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Define show to order records extracted from both source and target table. | |
<strong>If you do not specify any key column, then odb uses the Primary Key.</strong></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>output=[+]file</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Output file where the differences are reported. You can use stdout to print odb | |
output on the standard output. A <code>+</code> sign in front of the file-name tells odb to <strong>append</strong> to an existing file.<br> | |
<br> | |
Default value: <code>stdout</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>fs=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Field separator of the output file. You can define the field separator as follows:<br> | |
<br> | |
- Normal character (for example <code>fs=,</code>)<br> | |
- ASCII decimal (for example <code>fs=44</code> — 44 means comma)<br> | |
- ASCII octal value (for example <code>fs=054</code> — 054 means comma)<br> | |
- ASCII hex value (for example fs=<code>x2C</code> — x2C means comma)<br> | |
<br> | |
The default field separator is <code>,</code> (comma).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rs=<char>|<code></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Record separator used in the output file. You can define the | |
record separator the same way as the field separator.<br> | |
<br> | |
The default record separator is <code>\n</code> (new line).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>max=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The max number of records to compare. Default is to compare all records.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>rows=<num>|k<num>|m<num></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines the size of the I/O buffer for each extraction thread. You | |
can define the size of this buffer in two different ways:<br> | |
<br> | |
- number of rows (for example: <code>rows=100</code> means 100 rows as IO buffer)<br> | |
- buffer size in kB or MB (for example: <code>rows=k512</code> (512 kB buffer) or <code>rows=m20</code> (20MB buffer))</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ns=<nullstring></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">How odb represents NULL values in the output file. Default is the empty string (two field separators one after the other)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>es=<emptystring></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">How odb represents VARCHAR empty strings (NOT NULL with zero length) values in the output file. | |
Default is the empty string (two field separators one after the other)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nomark</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Don’t print the number of records extracted so far by each thread.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>soe</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Stop On Error. odb stops as soon as it encounters an error.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>parallel=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb uses as many <em>threads triplets</em> (extract from source, extract from target, compare) as the parallel argument.<br> | |
<br> | |
Each thread will take care of a specific range of the source table data defined through the <strong>splitby</strong> option.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>uncommitted</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb adds <code>FOR READ UNCOMMITTED ACCESS</code> to the select(s) command(s).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>splitby=<column></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Lets you to use parallel extract from any database. <strong><code><column></code> has to be a SINGLE, numeric column (or expression)</strong>. odb | |
calculates min()/max() value for <code><column></code> and assigns it to each <parallel> thread the extraction of the rows in its <em>bucket</em>.<br> | |
<br> | |
<strong>Example</strong><br> | |
<br> | |
<code>…:splitby=emp_id:parallel=4…</code><br> | |
<br> | |
with <code>min(emp_id)=1</code> and <code>max(emp_id)=1000</code>, the four threads will extract the following rows:<br> | |
<br> | |
<code>thread #0 emp_id >=1 and emp_id < 251</code><br> | |
<code>thread #1 emp_id >=251 and emp_id < 501</code><br> | |
<code>thread #2 emp_id >=501 and emp_id < 751</code><br> | |
<code>thread #3 emp_id >=751 and emp_id < 1001 (odb uses max(emp_id) + 1)</code><br> | |
<br> | |
If the values are not equally distributed, then data extraction is deskewed.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>print=[I][C][D]</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies which rows are printed in the output file:<br> | |
<br> | |
<code>I</code> prints the new rows on target. (Based on <strong>key</strong>.)<br> | |
<code>D</code> prints the missing rows on target. (Based on <strong>key</strong>.)<br> | |
<code>C</code> prints the source rows with the same <strong>key</strong> columns but differences in other fields.<br> | |
<br> | |
The default value for print is <code>IDC</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pre={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb runs a <strong>single instance</strong> of either a <code>sqlfile</code> script or <code>sqlcmd</code> SQL | |
command (enclosed between square brackets) on the <strong>target system</strong> immediately before reading the target table.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>post={@sqlfile}|{[sqlcmd]}</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb runs a <strong>single instance</strong> of either a <code>sqlfile</code> script or <code>sqlcmd</code> SQL | |
command (enclosed between square brackets) on the <strong>target system</strong> immediately after the target table has been compared.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>tpar=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of tables to compare in parallel when you have a list of tables in input.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>loaders=num</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb uses <code>num</code> load threads for each extract thread. Default is 2 loaders per extractor.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pwhere=<where condition></code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">This option is used in conjunction with parallel to <em>diff</em> only records | |
satisfying the where condition.<br> | |
<br> | |
For example: you want to compare rows with <code>TRANS_TS > 1999-12-12 09:00:00</code> from the source table | |
TRAFODION.MAURO.MFORDERS using eight parallel streams to a target table having the same name as the source table:<br> | |
<br> | |
<code>src=trafodion.mauro.mforders:tgt=trafodion.dest_schema.%t:parallel=8:pwhere=[TRANS_TS > TIMESTAMP ‘1999-12- 12 09:00:00’]…</code><br> | |
<br> | |
You can enclose the where condition between square brackets to avoid a misinterpretation of the characters in the where condition.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>quick</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Limits the comparison to the columns in the key option (PK by default). This is a fast way to check for new/missing records but | |
it will not find rows with differences in <em>non-key</em> columns.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>time</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">odb prints a <em>timeline</em>. (Milliseconds from starts)</p></td> | |
</tr> | |
</tbody> | |
</table> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_odb_as_a_query_driver_technology_preview">7. odb as a Query Driver (Technology Preview)</h2> | |
<div class="sectionbody"> | |
<div class="sect2"> | |
<h3 id="_getting_csv_output">7.1. Getting CSV Output</h3> | |
<div class="paragraph"> | |
<p>It’s often handy to get a CSV output ready to be imported into your spreadsheet while running performance tests. You can easily get this kind | |
of output with <code>–c</code> odb option.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ./odb64luo -u mauro -p xxx -d pglocal -x 3:"select count(*) from tpch.region" \ | |
-f 5:Q01.sql -f 3:Q02.sql -T 4 -q –c</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>This command runs:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Three copies of the <code>select count(<strong>): -x 3:”select count(</strong>) from tpch.region”</code></p> | |
</li> | |
<li> | |
<p>Five copies of <code>Q01.sql: -f 5:Q01.sql</code></p> | |
</li> | |
<li> | |
<p>Three copies of <code>Q02: -f 3:Q02.sql</code></p> | |
</li> | |
<li> | |
<p>Queuing the resulting 11 executions into four threads: <code>-T 4</code></p> | |
</li> | |
<li> | |
<p>Omitting query text and query results (<code>-q</code> is equivalent to <code>–q all</code>): <code>-q</code></p> | |
</li> | |
<li> | |
<p>Printing a CSV output: <code>–c</code></p> | |
</li> | |
</ul> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>The command produces the following output:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">odb [2011-12-12 08:08:43]: starting (4) threads... | |
Thread id,Proc id,Thread Exec#,Script | |
Cmd#,File,Label,Command,Rows,Rsds,Prepare(s),Exec(s),1st | |
Fetch(s),Fetch(s),Total(s),STimeline,ETimeline | |
1,1,0,0,(none),,"select count() from tpch.region",1,20,0.000,0.109,0.000,0.000,0.109,94,203 | |
0,0,0,0,(none),,"select count() from tpch.region",1,20,0.000,0.125,0.000,0.000,0.125,94,219 | |
2,2,0,0,(none),,"select count() from tpch.region",1,20,0.000,0.109,0.000,0.000,0.109,110,219 | |
2,6,1,0,Q01.sql,,"SELECT L_RETURNFLAG, L_LINESTATUS,SUM(L_QUANTITY)>",4,234,0.000,136.297,0.000,0.000,136.297,141,136438 | |
2,10,2,0,Q02.sql,,"SELECT S_ACCTBAL, S_NAME, N_NAME,P_PARTKEY,P_MF>",0,274,0.000,0.468,0.000,0.016,0.484,136438,136922 | |
0,4,1,0,Q01.sql,,"SELECT L_RETURNFLAG, L_LINESTATUS,SUM(L_QUANTITY)>",4,234,0.000,139.667,0.016,0.016,139.683,0,139683 | |
0,8,2,0,Q02.sql,,"SELECT S_ACCTBAL, S_NAME, N_NAME, P_PARTKEY,P_MFG>",0,274,0.000,0.015,0.000,0.000,0.015,139683,139698 | |
1,5,1,0,Q01.sql,,"SELECT L_RETURNFLAG, L_LINESTATUS,SUM(L_QUANTITY)>",4,234,0.000,144.347,0.015,0.015,144.362,141,144503 | |
1,9,2,0,Q02.sql,,"SELECT S_ACCTBAL, S_NAME, N_NAME, P_PARTKEY,P_MFG>",0,274,0.000,0.000,0.000,0.016,0.016,144503,144519 | |
3,3,0,0,Q01.sql,,"SELECT L_RETURNFLAG, L_LINESTATUS,SUM(L_QUANTITY)>",4,234,0.000,144.394,0.016,0.016,144.410,390,144800 | |
3,7,1,0,Q01.sql,,"SELECT L_RETURNFLAG, L_LINESTATUS,SUM(L_QUANTITY)>",4,234,0.000,69.373,0.000,0.000,69.373,144800,214173 | |
odb statistics: | |
Init timestamp: 2011-12-12 08:08:42 | |
Start timestamp: 2011-12-12 08:08:43 | |
End timestamp: 2011-12-12 08:12:17 | |
Elapsed [Start->End] (s): 214.173 | |
----</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p>The CSV output columns have the following meaning:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 30%;"> | |
<col style="width: 70%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Column</th> | |
<th class="tableblock halign-left valign-top">Meaning</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Thread ID</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Thread ID. Number of threads limited to 4 —> thread id values are 0, 1, 2, 3</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Proc ID</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Execution number. 11 executions in the 0-10 range.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Thread Exec#</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Progressive number (starting from 0) of execution for a specific thread.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script Cmd#</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">If your script contains multiple SQL statement, then they are numbered starting from zero.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>File</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Script file name or <code>(null)</code> for <code>–x</code> commands.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Label</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The label assigned though <code>set qlabel</code> in the scripts.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Command</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">First 30 characters of the SQL command. It will end with <code>></code> if the command text was truncated.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Rows</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of returned rows. Not printed if you used <code>–q</code>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Rsds</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Record Set Display Size. Gives you an idea of <em>how big</em> the result set is.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Prepare(s)</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Prepare (compile) time in seconds.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Exec(s)</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Execution time in seconds.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1st Fetch(s)</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Time needed to fetch the first row in seconds.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Fetch(s)</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Total Fetch time in seconds.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Total(s)</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Total query elapsed time from prepare to fetch in seconds.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Stimeline</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Queries start time line in milliseconds.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Etimeline</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Queries end time line in milliseconds.</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_assign_label_to_a_query">7.2. Assign Label to a Query</h3> | |
<div class="paragraph"> | |
<p>Sometimes it’s not easy to recognize a query by reading the first 30 characters. Therefore, odb lets you | |
assign a label to a generic query using:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">SET QLABEL <label></code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ cat script.sql | |
-- {project-name} TPC-H Query 1 SET QLABEL Q01 | |
SELECT | |
L_RETURNFLAG | |
, L_LINESTATUS | |
, SUM(L_QUANTITY) AS SUM_QTY | |
... | |
-- TPC-H/TPC-R Minimum Cost Supplier Query (Q2) | |
SET QLABEL Q02 | |
SELECT | |
S_ACCTBAL | |
, S_NAME | |
...</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Running this script includes the Query Label in the CSV output:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -u mauro -p xxx -d pglocal -f script.sql -q -c | |
odb [2011-12-12 09:06:28]: starting (1) threads... | |
Thread id,Proc id,Thread Exec#,Script | |
Cmd#,File,Label,Command,Rows,Rsds,Prepare(s),Exec(s),1st | |
Fetch(s),Fetch(s),Total(s),STimeline,ETimeline | |
0,0,0,0,script.sql,Q01,"SELECT L_RETURNFLAG, L_LINESTATUS, | |
SUM(L_QUANTITY)>",4,234,0.000,43.102,0.000,0.000,43.102,0,43102 | |
0,0,0,1,script.sql,Q02,"SELECT S_ACCTBAL, S_NAME, N_NAME, P_PARTKEY, | |
P_MFG>",0,274,0.000,0.016,0.000,0.000,0.016,43102,43118 | |
odb statistics: | |
Init timestamp: 2011-12-12 09:06:28 | |
Start timestamp: 2011-12-12 09:06:28 | |
End timestamp: 2011-12-12 09:07:11 | |
Elapsed [Start->End] (s): 43.118</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="query_driver_all_scripts_path">7.3. Run All Scripts With a Given Path</h3> | |
<div class="paragraph"> | |
<p>Using <code>-S <path></code> or <code>-P <path></code> options you can run all scripts with a given path | |
(for example, all files in a directory) either serially (<code>-S</code>) or in parallel (<code>-P</code>).</p> | |
</div> | |
<div class="paragraph"> | |
<p>Both options let you to use <strong><em>multiplying factors</em></strong> to run all scripts multiple times. | |
This multiplying factors are defined with a <code><number>:</code> preceding the script path.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Examples</strong></p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 40%;"> | |
<col style="width: 60%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">odb Command Line</th> | |
<th class="tableblock halign-left valign-top">Action</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>odb64luo -S ./test/queries/*.sql - c -q</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Executes <strong>serially</strong> all scripts with extension <code>.sql</code> under | |
<code>./test/queries/</code> providing CSV type output (<code>-c</code>) and omitting query output (<code>-q</code>).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>odb64luo -P test/queries/* -T 50 - c -q</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Runs <strong>in parallel</strong> all files under <code>test/queries/</code> using 50 threads | |
(ODBC connections) (<code>-T 50</code>), with CSV output (<code>-c</code>) and omitting query output (<code>-q</code>).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>odb64luo -P 3: test/queries/* -T 3 -c -q</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Runs <strong>in parallel three times (<code>3:</code>)</strong> all files under <code>test/queries/</code> | |
using three threads (ODBC connections) (<code>-T 3</code>), with CSV output (<code>-c</code>) and omitting query output (<code>-q</code>). | |
Scripts will be assigned to threads using <strong><em>standard assignment</em></strong>.</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>odb64luo -P -3: test/queries/* -T 3 -c -q</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Runs <strong>in parallel three times (<code>-3:</code>)</strong> all files under <code>test/queries/</code> | |
using three threads (ODBC connections) (<code>-T 3</code>), with CSV type output (<code>-c</code>) and omitting query output (<code>-q</code>). | |
Scripts will be assigned to threads using <strong><em>round-robin assignment</em></strong>.</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>To understand the difference between <strong>standard</strong> and <strong>round-robin</strong> assignments, imagine you have four scripts in | |
the target path. This is how the executions will be assigned to threads:</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 16%;"> | |
<col style="width: 14%;"> | |
<col style="width: 14%;"> | |
<col style="width: 14%;"> | |
<col style="width: 14%;"> | |
<col style="width: 14%;"> | |
<col style="width: 14%;"> | |
</colgroup> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"></td> | |
<th class="tableblock halign-center valign-top" colspan="3"><p class="tableblock">Standard Assignment (es. -P 3:)</p></th> | |
<th class="tableblock halign-center valign-top" colspan="3"><p class="tableblock">Round-Robin Assignment (es. -P -3:)</p></th> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"></td> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Thread 1</p></th> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Thread 2</p></th> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Thread 3</p></th> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Thread 1</p></th> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Thread 2</p></th> | |
<th class="tableblock halign-left valign-top"><p class="tableblock">Thread 3</p></th> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">nth execution</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">…</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">…</p></td> | |
<td class="tableblock halign-left valign-top"></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">…</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">…</p></td> | |
<td class="tableblock halign-left valign-top"></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">4th execution</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script4.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script4.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>…</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script3.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>…</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">3rd execution</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script3.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script3.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script3.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script3.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script4.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">2nd execution</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script4.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">1st execution</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script1.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script2.sql</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Script3.sql</code></p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_randomizing_execution_order">7.4. Randomizing Execution Order</h3> | |
<div class="paragraph"> | |
<p>You can use the <code>-Z</code> option to <em>shuffle</em> the odb internal execution table. | |
This way the execution order is not predictable.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Examples</strong></p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 45%;"> | |
<col style="width: 55%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">odb Command Line</th> | |
<th class="tableblock halign-left valign-top">Action</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>odb64luo… -S 3: test/queries/* -Z -c –q</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Executes three times (<code>3:</code>) all files in the <code>test/queries</code> directory | |
serially (<code>-S</code>) and in random order (<code>-Z</code>).</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>odb64luo… -P 3: test/queries/* -Z –T 5 - c -q</code></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Executes three times (<code>3:</code>) all files in the <code>test/queries</code> directory in | |
parallel (<code>-P</code>), using five threads (<code>-T 5</code>) and in random order (<code>-Z</code>).</p></td> | |
</tr> | |
</tbody> | |
</table> | |
</div> | |
<div class="sect2"> | |
<h3 id="_defining_a_timeout">7.5. Defining a Timeout</h3> | |
<div class="paragraph"> | |
<p>You can stop odb after a given timeout (assuming the execution is not already completed) using <code>-maxtime <seconds></code> option.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/Devel/odb $ ./odb64luo -S /home/mauro/scripts/*.sql –maxtime 7200</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The command executes, <strong>serially,</strong>( all scripts with extension <code>.sql</code> under | |
<code>/home/mauro/scripts/</code>; if the execution is not completed after two hours (7200 seconds), then odb stops.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_simulating_user_thinking_time">7.6. Simulating User Thinking Time</h3> | |
<div class="paragraph"> | |
<p>You can simulate user <strong><em>thinking time</em></strong> using the <code>-ttime <delay></code> option. | |
This argument introduces a <code><delay></code> millisecond pause between two consecutive executions in the same thread.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ ./odb64luo -f 5:script1.sql -c -q -ttime 75 -T 2</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>This command runs five times <code>script1.sql</code> using two threads. Each thread waits 75 milliseconds before starting | |
the next execution within a thread. You can also use a <strong><em>random thinking time</em></strong> in a given <code>min:max</code> range.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p>The following command starts commands within a thread with a random delay between 50 and 500 milliseconds:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ ./odb64luo -f 5:script1.sql -c -q -ttime 50:500 -T 2</code></pre> | |
</div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_starting_threads_gracefully">7.7. Starting Threads Gracefully</h3> | |
<div class="paragraph"> | |
<p>You might want to wait a little before starting the next thread. This can be obtained using the <code>-delay</code> option.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ ./odb64luo -f 5:script1.sql -c -q -delay 200 -T 2</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>This command runs five times <code>script1.sql</code> using two threads. Each thread will be started 200 milliseconds after the other.</p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
<code>-delay</code> introduces a delay during threads start-up while <code>–ttime</code> introduces a delay between one command and another within the same | |
thread. | |
</td> | |
</tr> | |
</table> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_re_looping_a_given_workload">7.8. Re-looping a Given Workload</h3> | |
<div class="paragraph"> | |
<p>Using <code>-L</code> option you can re-loop the workload defined through <code>-x</code>, <code>-f</code>, <code>-P</code>, and <code>-S</code> commands a given number of times. | |
Each thread will re-loop the same number of times.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">~/src/C/odb $ *./*odb64luo -f 5:script1.sql -c -q -M 75 -T 2 -L 3</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>re-loops three times (<code>-L 3</code>) the same five executions, using two threads (<code>-T 2</code>) with a 75 millisecond pause (<code>-M 75</code>) between two | |
consecutive executions in the same thread.</p> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_odb_as_a_sql_interpreter_technology_preview">8. odb as a SQL Interpreter (Technology Preview)</h2> | |
<div class="sectionbody"> | |
<div class="paragraph"> | |
<p>To start the odb SQL Interpreter you have to use <code>-I</code> (uppercase i) switch with an optional argument.</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ odb64luo –u user –p xx –d dsn –I MFTEST</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>The optional <code>-I</code> argument (<code>MFTEST</code> in this example) is used to specify the <code><strong>.odbrc</strong></code> section containing | |
commands to be automatically executed when odb starts. See <sql_run_commands, Run Commands When Interpreter Starts>>.</p> | |
</div> | |
<div class="sect2"> | |
<h3 id="_main_odb_sql_interpreter_features">8.1. Main odb SQL Interpreter Features</h3> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p><strong>It uses <code>mreadline</code> library</strong> to manage command line editing and history. History will keep track of | |
the whole <strong>command</strong>, not just… lines: if you enter a SQL command in more than one line:</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">S01_Maurizio@TRAFODION64[MFTEST]SQL> select | |
S01_Maurizio@TRAFODION64[MFTEST]...> count() | |
S01_Maurizio@TRAFODION64[MFTEST]...> from | |
S01_Maurizio@TRAFODION64[MFTEST]...> t1;</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>When you press the up arrow key the whole command (up to semi-colon) will be ready for editing and/or re-run.</p> | |
</div> | |
<div class="paragraph"> | |
<p><code>mreadline</code> provides several useful extra features:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>CTRL-V</strong> to edit the current command using your preferred editor (<code>$EDITOR</code> is used). When the editing session is closed the current | |
command is automatically updated.</p> | |
</li> | |
<li> | |
<p><strong>CTRL-U/CTRL-L</strong> to change the command case.</p> | |
</li> | |
<li> | |
<p><strong>CTRL-X</strong> to kill the current command.</p> | |
</li> | |
<li> | |
<p>See on-line help for the other <code>mreadline</code> commands.</p> | |
<div style="page-break-after: always;"></div> | |
</li> | |
</ul> | |
</div> | |
</li> | |
<li> | |
<p><strong>History is saved</strong> when you exit the SQL Interpreter in a file identified by the <code>ODB_HIST</code> environment variable. | |
You can change the number of commands saved in the history file (default <code>100</code>):</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">S01_Maurizio@TRAFODION64[MFTEST]SQL> SET HIST 200</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p><strong>Customizable prompt</strong>. You can personalize your prompt through the set prompt command. | |
Under Unix/Linux/Cygwin you can use the standard ANSI codes to create color prompts. | |
See <a href="#sql_custom_prompt">Customize Interpreter Prompt</a>.</p> | |
</li> | |
<li> | |
<p><strong>Multi-threaded odb instances</strong> can be run from within the single-threaded Interpreter with the <code>odb</code> keyword. | |
This runs another odb instance using the same credentials, data source, and connection attributes used to start the interpreter:</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">S01_Maurizio@TRAFODION64[MFTEST]SQL> odb -l | |
src=myfile:tgt=mytable:parallel=8:... | |
S01_Maurizio@TRAFODION64[MFTEST]SQL> odb -e | |
src=mytable:tgt=myfile:parallel=8:...</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p><strong>Define Aliases</strong> with parameter substitution.</p> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">root@MFDB[MFDB]SQL> SET ALIAS count "SELECT ROW COUNT FROM &1;"</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>When you call the alias <code>count</code> the first argument will be substituted to <code>&1</code>. | |
You can use <strong>up to nine</strong> positional parameters (<code>&1</code> to <code>&9</code>).</p> | |
</div> | |
</li> | |
<li> | |
<p>You can <strong>run operating system commands</strong> with <code>!command</code>.</p> | |
</li> | |
<li> | |
<p>You can run scripts with <code>@script</code>.</p> | |
</li> | |
<li> | |
<p>You can spool to file with <code>set spool <myfile></code> and stop spooling with <code>set spool off</code>.</p> | |
</li> | |
<li> | |
<p>You can switch to a special <em>prepare only</em> mode with <code>set prepare on</code>. This way, commands you type | |
will be just prepared, not executed.</p> | |
<div style="page-break-after: always;"></div> | |
</li> | |
<li> | |
<p>Different databases use different commands to set default schema(s):</p> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>Trafodion: <code>set schema <name>;</code></p> | |
</li> | |
<li> | |
<p>MySQL: <code>use <name>;</code></p> | |
</li> | |
<li> | |
<p>PostgreSQL/Vertica: <code>set search_path to <name1,name2,…>;</code></p> | |
</li> | |
<li> | |
<p>Teradata: <code>set database <name>;</code></p> | |
<div class="paragraph"> | |
<p><code>set chsch <command></code> is used to define database specific commands to change your schema. When odb recognize the | |
<code>change schema</code> command it will update accordingly internal catalog (if any) and schema names.</p> | |
</div> | |
</li> | |
</ul> | |
</div> | |
</li> | |
<li> | |
<p>To list database objects, you can use <code>ls</code> command.</p> | |
<div class="paragraph"> | |
<p><strong>Examples</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">S01_Maurizio@MFTEST[MFTEST]SQL> ls . # list all objects in the current | |
schema | |
TABLE : CITIES | |
TABLE : CUSTOMER | |
TABLE : LINEITEM | |
TABLE : NATION | |
TABLE : ORDERS | |
TABLE : PART | |
TABLE : PARTSUPP | |
TABLE : REGION | |
TABLE : SUPPLIER | |
TABLE : T1 | |
VIEW : V_CITIES | |
S01_Maurizio@MFTEST[MFTEST]SQL> ls -t %S << list tables (-t) ending with S CITIES ORDERS | |
S01_Maurizio@MFTEST[MFTEST]SQL> ls -v << list views (-v) V_CITIES | |
S01_Maurizio@MFTEST[MFTEST]SQL> ls -s << list schemas (-s) | |
... and so on ...</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</li> | |
<li> | |
<p>To get tables DDL, you can use either <code>ls –T <table></code> or <code>ls -D <table></code>.</p> | |
<div class="paragraph"> | |
<p><strong>Examples</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">mauro pglocal[PUBLIC] (09:12:56) SQL> ls -T tpch.orders | |
Describing: postgres.TPCH.orders | |
+---------------+-------------+----+-------+---------------+ | |
|COLUMN |TYPE |NULL|DEFAULT|INDEX | | |
+---------------+-------------+----+-------+---------------+ | |
|o_orderkey |int8 |NO | |orders_pkey 1 U| | |
|o_custkey |int8 |NO | | | | |
|o_orderstatus |bpchar(1) |NO | | | | |
|o_totalprice |numeric(15,2)|NO | | | | |
|o_orderdate |date |NO | | | | |
|o_orderpriority|bpchar(15) |NO | | | | |
|o_clerk |bpchar(15) |NO | | | | |
|o_shippriority |int4 |NO | | | | |
|o_comment |varchar(80) |NO | | | | |
+---------------+-------------+----+-------+---------------+ | |
mauro pglocal[PUBLIC] (09:13:20) SQL> ls -D tpch.orders | |
CREATE TABLE postgres.TPCH.orders ( o_orderkey int8 | |
,o_custkey int8 | |
,o_orderstatus bpchar(1) | |
,o_totalprice numeric(15,2) | |
,o_orderdate date | |
,o_orderpriority bpchar(15) | |
,o_clerk bpchar(15) | |
,o_shippriority int4 | |
,o_comment varchar(80) | |
,primary key (o_orderkey) | |
);</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>You can <strong>define your own variables</strong> or use odb internal variables or environment variables directly from the Interpreter.</p> | |
<div style="page-break-after: always;"></div> | |
</li> | |
<li> | |
<p>You can <code>set pad fit</code> to <strong><em>automatically shrink CHAR/VARCHAR fields in order to fit one record in one line</em></strong>. | |
Line length is defined through <code>set scols #</code>. Each record will be printed in one line truncating the length of CHAR/VARCHAR | |
fields proportionally to their original display size length. In case of field truncation a <code>></code> character will be printed | |
at the end of the truncated string.</p> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">MFELICI [MAURIZIO] (03:30:32) SQL> select [first 5] from part; | |
P_PARTKEY|P_NAME |P_MFGR |P_BRAND|P_TYPE | P_SIZE |P_CONTAINER|P_RETAILPRICE|P_COMMENT | |
---------+-------------------------------+---------------+-------+---------------+----------+-------+----------------+-------------- | |
33 |maroon beige mint cyan peru |Manufacturer#2>|Brand#>|ECONOMY PLATED>| 16|LG PKG>| 933.03|ly eve | |
39 |rose dodger lace peru floral |Manufacturer#5>|Brand#>|SMALLPOLISHED> | 43|JUMBO >| 939.03|se slowly abo> | |
60 |sky burnished salmon navajo hot|Manufacturer#1>|Brand#>|LARGE POLISHED>| 27|JUMBO >| 960.06| integ | |
81 |misty salmon cornflower dark f>|Manufacturer#5>|Brand#>|ECONOMY BRUSHE>| 21|MED BA>| 981.08|ove the furious | |
136 |cornsilk blush powder tan rose |Manufacturer#2>|Brand#>|SMALL PLATED S>| 2|WRAP B>| 1036.13|kages print c></code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>You can <code>set plm</code> to print one field per row. This is useful when you have to carefully analyze few records.</p> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">MFELICI [MAURIZIO] (03:38:12) SQL> SET PLM ON | |
MFELICI [MAURIZIO] (03:38:12) SQL> select * from part where p_partkey =136; | |
P_PARTKEY 136 | |
P_NAME :cornsilk blush powder tan rose P_MFGR :Manufacturer#2 | |
P_BRAND :Brand#22 | |
P_TYPE :SMALL PLATED STEEL | |
P_SIZE 2 | |
P_CONTAINER :WRAP BAG P_RETAILPRICE:1036.13 | |
P_COMMENT :kages print carefully</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>Check the rest on your own.</p> | |
</li> | |
</ol> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="sect3"> | |
<h4 id="_odb_sql_interpreter_help">8.1.1. odb SQL Interpreter help</h4> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">`mauro pglocal[PUBLIC] (06:51:20) SQL>` *help | |
All the following are case insensitive: | |
h | help : print this help | |
i | info : print database info | |
q | quit : exit SQL Interpreter | |
c | connect { no | [user[/pswd][;opts;&#8230;] (re/dis)connect using previous or new user | |
odb odb_command : will run an odb instance using the same DSN/credentials | |
ls -[type] [pattern] : list objects. Type=(t)ables, (v)iews, s(y)nonyns, (s)chemas | |
: (c)atalogs, syst(e)m tables, (l)ocal temp, (g)lobal temp | |
: (m)at views, (M)mat view groups, (a)lias, (A)ll object types | |
: (D)table DDL, (T)table desc | |
print <string> : print <string> | |
!cmd : execute the operating system cmd | |
@file [&0]...[&9] : execute the sql script in file | |
set : show all settings | |
set alias [name] [cmd|-]: show/set/change/delete aliases | |
set chsch [cmd] : show/set change schema command | |
set cols [#cols] : show/set ls number of columns | |
set cwd [<directory>] : show/set current working directory | |
set drs [on|off] : show/enable/disable describe result set mode | |
set fs [<char>] : show/set file field separator | |
set hist [#lines] : show/set lines saved in the history file | |
set maxfetch [#rows] : show/set max lines to be fetched (-1 = unlimited) | |
set nocatalog [on|off] : show/enable/disable "no catalog" database mode) | |
set nocatnull [on|off] : show/enable/disable "no catalog as null" database mode) | |
set noschema [on|off] : show/enable/disable "no schema" database mode) | |
set nullstr [<string>] : show/set string used to display NULLs ( to make it Null) | |
set pad [fit|full|off] : show/set column padding | |
set param name [value|-]: show/set/change/delete a parameter | |
set pcn [on|off] : show/enable/disable printing column names | |
set plm [on|off] : show/enable/disable print list mode (one col/row) | |
set prepare [on|off] : show/enable/disable 'prepare only' mode | |
set prompt [string] : show/set prompt string | |
set query_timeout [s] : show/set query timeout in seconds (def = 0 no timeout) | |
set quiet [cmd|res|all|off] : show/enable/disable quiet mode | |
set rowset [#] : show/set rowset used to fetch rows | |
set soe [on|off] : show/enable/disable Stop On Error mode | |
set spool [<file>|off] : show/enable/disable spooling output on <file> | |
<SQL statement>; : everything ending with ';' is sent to the database | |
mreadline keys: | |
Control-A : move to beginning of line Control-P : history Previous | |
Control-E : move to end of line Up Arrow : history Previous | |
Control-B : move cursor Back Control-N : history Next | |
Left Arrow : move cursor Back Down Arrow : history Next | |
Control-F : move cursor Forward Control-W : history List | |
Right Arrow: move cursor Forward Control-R : Redraw | |
Control-D : input end (exit) - DEL right Control-V : Edit current line | |
Control-L : Lowercase Line Control-X : Kill line | |
Control-U : Uppercase Line # Control-G : load history entry #</code></pre> | |
</div> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
</div> | |
<div class="sect2"> | |
<h3 id="sql_run_commands">8.2. Run Commands When the Interpreter Starts</h3> | |
<div class="paragraph"> | |
<p>When the odb SQL Interpreter starts it looks for the <strong><em>Initialization File</em></strong>. | |
This Initialization File is made of <strong><em>Sections</em></strong> containing the commands to be executed.</p> | |
</div> | |
<div class="paragraph"> | |
<p>To find the Initialization File, odb checks the <code>ODB_INI</code> environment variable. If this variable is not set, | |
then odb looks for a file named <code>.odbrc</code> (*nix) or <code>_odbrc</code> (Windows) under your HOME directory.</p> | |
</div> | |
<div class="paragraph"> | |
<p>The <strong><em>Initialization File</em></strong> contains <strong><em>Sections</em></strong> identified by names between square brackets. | |
For example, the following section is named <code>MFTEST</code>:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">[MFTEST] | |
set pcn on | |
set pad fit | |
set fs | | |
set cols 3 30 | |
set editor "vim -n --noplugin" | |
set efile /home/felici/.odbedit.sql set prompt "%U %D [%S] (%T) %M> " | |
set alias count "select row count from &1;" | |
set alias size "select sum(current_eof) from table (disk label statistics (&1) );" | |
set alias ll "select left(object_name, 40) as object_name, sum(row_count) as nrows, | |
count(partition_num) as Nparts, sum(current_eof) as eof from table(disk label statistics( | |
using (select from (get tables in schema &catalog.&schema, no header, return full names) | |
s(b) ))) group by object_name order by object_name;" | |
set schema TRAFODION.MAURIZIO;</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>the odb SQL Interpreter automatically runs all commands in the section identified by the <code>-I</code> argument (for example <code>-I MFTEST</code>). | |
A section named <code>DEFAULT</code> will be executed when <code>-I</code> has no arguments.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="sql_custom_prompt">8.3. Customizing the Interpreter Prompt</h3> | |
<div class="paragraph"> | |
<p>You can define your prompt through the <code>set prompt</code> command when running the SQL Interpreter. <code>set prompt</code> can be | |
executed interactively or included in your (<code>$ODB_INI</code>) <strong><em>Initialization File</em></strong>. <code>set prompt</code> recognizes and expands | |
the following variables:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><code><strong>%U</strong></code> —> User name</p> | |
</li> | |
<li> | |
<p><code><strong>%D</strong></code> —> Data Source name</p> | |
</li> | |
<li> | |
<p><code><strong>%S</strong></code> —> Schema name</p> | |
</li> | |
<li> | |
<p><code><strong>%T</strong></code> —> Current Time</p> | |
</li> | |
<li> | |
<p><code><strong>%M</strong></code> —> odb mode:</p> | |
<div class="paragraph"> | |
<p><code>SQL</code> when running sql commands</p> | |
</div> | |
<div class="paragraph"> | |
<p><code>PRE</code> if you’re in "prepare only" mode</p> | |
</div> | |
<div class="paragraph"> | |
<p><code>SPO</code> if you are spooling output somewhere</p> | |
</div> | |
<div class="paragraph"> | |
<p><code>NDC</code> (No Database Connection)</p> | |
</div> | |
</li> | |
</ul> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">SET PROMPT "Prompt for %U connected via %D to %S in %M mode > "</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Generates the following prompt:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">Prompt for S01_Maurizio connected via CIV to CIV03 in SQL mode ></code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Under Cygwin, Unix and Linux (and probably under Windows too using ANSI.SYS driver - not tested), | |
you can use standard ANSI escape color codes.</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">set prompt "\^A^[[01;32m\^A%U@%D^A\^[[01;34m^A[%S]\^A^[[00m\ ^A (%T) %M> "</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Where:</p> | |
</div> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p><strong>^A</strong> is a <em>real</em> Control-A (ASCII 001 and 002) before and after each color code sequence.</p> | |
</li> | |
<li> | |
<p><strong>^[</strong> is a <em>real</em> Escape Character. The meaning of the ANSI color codes are:</p> | |
<div class="paragraph"> | |
<p><strong>^[[01;32m</strong> —> green</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>^[[01;34m</strong> —> blue</p> | |
</div> | |
<div class="paragraph"> | |
<p><strong>^[[00m</strong> —> reset.</p> | |
</div> | |
</li> | |
</ol> | |
</div> | |
<div class="paragraph"> | |
<p><strong>Example Prompt</strong></p> | |
</div> | |
<div class="paragraph"> | |
<p><span class="image"><img src="images/sql_ansi_colors.jpg" alt="image"></span></p> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div class="sect1"> | |
<h2 id="_appendixes">9. Appendixes</h2> | |
<div class="sectionbody"> | |
<div class="sect2"> | |
<h3 id="_a_troubleshooting">9.1. A. Troubleshooting</h3> | |
<div class="olist arabic"> | |
<ol class="arabic"> | |
<li> | |
<p>odb uses Condition Variables to synchronize threads during copy and parallel load operations.</p> | |
</li> | |
<li> | |
<p>Most of the memory allocation operations are dynamic. For example, you can execute an SQL | |
command as long as you want. However, you can <em>hard code</em> limits as follows:</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="cplusplus"><span class="preprocessor">#define</span> MAX_VNLEN <span class="integer">32</span> <span class="comment">/* Max variable name length */</span> | |
<span class="preprocessor">#define</span> MAXCOL_LEN <span class="integer">128</span> <span class="comment">/* Max column name length */</span> | |
<span class="preprocessor">#define</span> MAXOBJ_LEN <span class="integer">128</span> <span class="comment">/* Max catalog/schema/table name length */</span> | |
<span class="preprocessor">#define</span> MAX_CLV <span class="integer">64</span> <span class="comment">/* Max command line variables (-var) */</span></code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>Some Linux/UNIX systems (notably the Linux Loader) have huge default stack size. Due to this | |
extremely large value, you can have errors like this when starting tens/hundreds of threads:</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">Error starting cmd thread #: cannot allocate memory</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>If you get this error, then check your default stack size:</p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ulimit -a | |
core file size (blocks, -c) 0 | |
data seg size (kbytes, -d) unlimited | |
max nice (-e) 0 | |
file size (blocks, -f) unlimited | |
pending signals (-i) 137215 | |
max locked memory (kbytes, -l) 32 | |
max memory size (kbytes, -m) unlimited | |
open files (-n) 65536 | |
pipe size (512 bytes, -p) 8 | |
POSIX message queues (bytes, -q) 819200 | |
max rt priority (-r) 0 | |
stack size (kbytes, -s) 204800 | |
cpu time (seconds, -t) unlimited | |
max user processes (-u) 2047 | |
virtual memory (kbytes, -v) unlimited | |
file locks (-x) unlimited</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>In the example above, the <code>stack size</code> value is the problem. Reset it to a reasonable value. (The value will be reset to the initial value when you start a new session).</p> | |
</div> | |
<div style="page-break-after: always;"></div> | |
<div class="paragraph"> | |
<p><strong>Example</strong></p> | |
</div> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">$ ulimit -s 4096</code></pre> | |
</div> | |
</div> | |
</li> | |
<li> | |
<p>If you get errors such as:</p> | |
<div class="listingblock"> | |
<div class="content"> | |
<pre class="CodeRay highlight"><code data-lang="text">C:\Users\felici> odb64luo -u xx -p yy -d oraxe -l src=region.tbl.gz:tgt=region:fs=^|:truncate | |
odb [2012-10-11 13:27:22]: starting ODBC connection(s)... 0 | |
[0] odb(5020) - [Oracle][ODBC]Optional feature not implemented. (State: HYC00 Native Err: 0)</code></pre> | |
</div> | |
</div> | |
<div class="paragraph"> | |
<p>Try adding <code>-nocatnull</code> to your command line. When the back-end database doesn’t use catalogs, | |
then you should use an <strong>empty string</strong> as catalog name.</p> | |
</div> | |
<div class="admonitionblock note"> | |
<table> | |
<tr> | |
<td class="icon"> | |
<i class="fa icon-note" title="Note"></i> | |
</td> | |
<td class="content"> | |
Some flawed ODBC Drivers unfortunately want NULL here — instead of <strong>empty strings</strong> as it should be. | |
</td> | |
</tr> | |
</table> | |
</div> | |
</li> | |
<li> | |
<p>You can have errors loading <code>TIME(N)</code> fields with` N>0` under Trafodion because the | |
ODBC Driver does not manage the field display size when <code>N>0</code>.</p> | |
</li> | |
<li> | |
<p>If you have problems starting odb on Unix/Linux check:</p> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p>The shared library dependencies with <code>ldd <odb_executable_name></code>.</p> | |
</li> | |
<li> | |
<p>The shared lib path defined in the following environment variables used by the shared library loader:</p> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>Linux</strong>: <code>LD_LIBRARY_PATH</code></p> | |
</li> | |
<li> | |
<p><strong>IBM IAX</strong>: <code>LIBPATH</code> (not currently supported)</p> | |
</li> | |
<li> | |
<p><strong>HP/UX</strong>: <code>SHLIB_PATH</code> (not currently supported)</p> | |
</li> | |
</ul> | |
</div> | |
</li> | |
</ul> | |
</div> | |
</li> | |
</ol> | |
</div> | |
<div style="page-break-after: always;"></div> | |
</div> | |
<div class="sect2"> | |
<h3 id="_b_develop_and_test_odb">9.2. B. Develop and Test odb</h3> | |
<div class="sect3"> | |
<h4 id="_develop">9.2.1. Develop</h4> | |
<div class="paragraph"> | |
<p>odb is coded in "ANSI C" (K&R programming style) and is compiled in a 64-bit version on the Linux platform, linked | |
to the unixODBC driver manager. Other platforms, compilers, and ODBC libraries have not yet been tested.</p> | |
</div> | |
<table class="tableblock frame-all grid-all spread"> | |
<colgroup> | |
<col style="width: 16.6666%;"> | |
<col style="width: 16.6666%;"> | |
<col style="width: 44.4444%;"> | |
<col style="width: 22.2224%;"> | |
</colgroup> | |
<thead> | |
<tr> | |
<th class="tableblock halign-left valign-top">Platform</th> | |
<th class="tableblock halign-left valign-top">Compiler</th> | |
<th class="tableblock halign-left valign-top">ODBC Libraries</th> | |
<th class="tableblock halign-left valign-top">Note</th> | |
</tr> | |
</thead> | |
<tbody> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Linux</strong></p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">gcc</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">unixODBC (supported), iODBC (not currently supported), Data Direct (not currently supported), Teradata (not currently supported)</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">64 bit (32 bit is not currently supported)</p></td> | |
</tr> | |
<tr> | |
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>MS-Windows</strong><br> | |
(not tested)</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">Visual Studio (not tested)</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">MS-Windows (not tested)</p></td> | |
<td class="tableblock halign-left valign-top"><p class="tableblock">64 bit</p></td> | |
</tr> | |
</tbody> | |
</table> | |
<div class="paragraph"> | |
<p>C compilers are set with “all warnings” enabled and odb has to compile, on each platform, with no errors (of course) AND no warnings. Tools used | |
to code odb:</p> | |
</div> | |
<div class="ulist"> | |
<ul> | |
<li> | |
<p><strong>vim</strong> (<a href="http://www.vim.org" class="bare">http://www.vim.org</a>) as editor (or Visual Studio embedded editor)</p> | |
</li> | |
<li> | |
<p><strong>splint</strong> (<a href="http://www.splint.org" class="bare">http://www.splint.org</a>) to statically check the source code</p> | |
</li> | |
</ul> | |
</div> | |
</div> | |
<div class="sect3"> | |
<h4 id="_test">9.2.2. Test</h4> | |
<div class="paragraph"> | |
<p>The info, load, extract, and copy operations of odb have been fully tested. In addition, odb had been tested using a set of | |
137 standard tests to check functionalities and identify memory/thread issues.</p> | |
</div> | |
</div> | |
</div> | |
</div> | |
</div> | |
</div> | |
<div id="footer"> | |
<div id="footer-text"> | |
Version 2.3.0<br> | |
Last updated 2018-03-12 18:11:57 CST | |
</div> | |
</div> | |
<script> | |
var _gaq = _gaq || []; | |
_gaq.push(['_setAccount','UA-72491210-1']); | |
_gaq.push(['_trackPageview']); | |
(function() { | |
var ga = document.createElement('script'); | |
ga.type = 'text/javascript'; | |
ga.async = true; | |
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; | |
var s = document.getElementsByTagName('script')[0]; | |
s.parentNode.insertBefore(ga, s); | |
})(); | |
</script> | |
</body> | |
</html> |