| <?xml version="1.0"?> |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <!DOCTYPE document |
| [ |
| <!ENTITY sect-num '18'> |
| <!ENTITY hellip "…" > |
| ]> |
| |
| <document prev="realtime-results.html" next="boss.html" id="$Id$"> |
| |
| <properties> |
| <title>User's Manual: Best Practices</title> |
| </properties> |
| |
| <body> |
| |
| <section name="§-num;. Best Practices" anchor="best_practices"> |
| </section> |
| |
| <section name="§-num;.1 Always use latest version of JMeter" anchor="use_latest_version"> |
| <p>The performance of JMeter is being constantly improved, so users are highly encouraged to use the most up to date version. <br/> |
| Ensure you always read <a href="../changes.html">changes list</a> to be aware of new improvements and components. |
| You should absolutely avoid using versions that are older than 3 versions before the last one. |
| </p> |
| </section> |
| |
| <section name="§-num;.2 Use the correct Number of Threads" anchor="sizing_threads"> |
| <p>Your hardware capabilities as well as the Test Plan design will both impact the number of threads you can effectively |
| run with JMeter. The number will also depend on how fast your server is (a faster server |
| makes JMeter work harder since it returns a response quicker). As with any Load Testing tool, if you don't correctly size |
| the number of threads, you will face the "Coordinated Omission" problem which can give you wrong or inaccurate results. |
| If you need large-scale load testing, consider running multiple non-GUI JMeter instances on multiple machines |
| using distributed mode (or not). When using distributed mode the result file is combined on the Controller node, if |
| using multiple autonomous instances, the sample result files can be combined for subsequent analysis. |
| For testing how JMeter performs on a given platform, the JavaTest sampler can be used. |
| It does not require any network access so can give some idea as to the maximum throughput achievable. |
| </p> |
| <p> |
| JMeter has an option to delay thread creation until the thread starts sampling, i.e. after any thread group delay and the ramp-up time for the thread itself. |
| This allows for a very large total number of threads, provided that not too many are active concurrently. |
| </p> |
| </section> |
| |
| <section name="§-num;.3 Where to Put the Cookie Manager" anchor="put_cookie_manager"> |
| <p>See <a href="build-web-test-plan.html#adding_cookie_support">Building a Web Test</a> |
| for information.</p> |
| </section> |
| |
| <section name="§-num;.4 Where to Put the Authorization Manager" anchor="put_auth_manager"> |
| <p>See <a href="build-adv-web-test-plan.html#header_manager">Building an Advanced |
| Web Test</a> for information.</p> |
| </section> |
| |
| <section name="§-num;.5 Using the HTTP(S) Test Script Recorder" anchor="proxy_server"> |
| <p>Refer to <complink name="HTTP(S) Test Script Recorder"/> for details on setting up the |
| recorder. The most important thing to do is filter out all requests you aren't |
| interested in. For instance, there's no point in recording image requests (JMeter can |
| be instructed to download all images on a page - see <complink name="HTTP Request"/>). |
| These will just clutter your test plan. Most likely, there is an extension all your files |
| share, such as <code>.jsp</code>, <code>.asp</code>, <code>.php</code>, <code>.html</code> or the like. |
| These you should "<code>include</code>" by entering "<code>.*\.jsp</code>" as an "Include Pattern". </p> |
| <p>Alternatively, you can exclude images by entering "<code>.*\.gif</code>" as an "Exclude Pattern". |
| Depending on your application, this may or may not be a better way to go. You may |
| also have to exclude stylesheets, javascript files, and other included files. Test |
| out your settings to verify you are recording what you want, and then erase and start |
| fresh.</p> |
| <!--TODO: most uses for include/exclude are now |
| covered by the Grouping features. include/exclude may still be useful to filter hosts out, so |
| that regular browsing with the proxy enabled and the browser configured won't fill your test |
| plan with crap. Think about other uses. --> |
| <p>The HTTP(S) Test Script Recorder expects to find a ThreadGroup element with a Recording Controller |
| under it where it will record HTTP Requests to. This conveniently packages all your samples under one |
| controller, which can be given a name that describes the test case.</p><!--TODO: is this best practice? I don't think so!--> |
| <p>Now, go through the steps of a Test Case. If you have no pre-defined test cases, use |
| JMeter to record your actions to define your test cases. Once you have finished a |
| definite series of steps, save the entire test case in an appropriately named file. Then, wipe |
| clean and start a new test case. By doing this, you can quickly record a large number of |
| test case "rough drafts".</p><!--TODO: no need to wipe out now - you can create a new controller and |
| resume recording into it--> |
| <p>One of the most useful features of the HTTP(S) Test Script Recorder is that you can abstract out |
| certain common elements from the recorded samples. By defining some |
| <a href="functions.html">user-defined variables</a> at the Test Plan level or in |
| <complink name="User Defined Variables"/> elements, you can have JMeter automatically |
| replace values in you recorded samples. For instance, if you are testing an app on |
| server "<code>xxx.example.com</code>", then you can define a variable called "<code>server</code>" with the value of |
| "<code>xxx.example.com</code>", and anyplace that value is found in your recorded samples will be replaced |
| with "<code>${server}</code>". |
| |
| <note>Please note that matching is case-sensitive.</note> |
| |
| </p> |
| <p> |
| If JMeter does not record any samples, check that the browser really is using the proxy. |
| If the browser works OK even if JMeter is not running, then the browser cannot be using the proxy. |
| Some browsers ignore proxy settings for <code>localhost</code> or <code>127.0.0.1</code>; try using the local hostname or IP instead. |
| </p> |
| <p> |
| The error "<code>unknown_ca</code>" probably means that you are trying to record HTTPS, and the browser has not accepted the |
| JMeter Proxy server certificate. |
| </p> |
| <!--TODO: Describe how to use this to record against one server[name] |
| and run against another. Actually, this particular example may be beter for the HTTP Request Defaults… |
| a good similar example for the variables is http vs. https.--> |
| <!--SUMMARY of all TODO's above: we need some actual usage to identify best practices |
| with this new features.--> |
| </section> |
| |
| <section name="§-num;.6 User variables" anchor="user_variables"> |
| <p> |
| Some test plans need to use different values for different users/threads. |
| For example, you might want to test a sequence that requires a unique login for each user. |
| This is easy to achieve with the facilities provided by JMeter. |
| </p> |
| <p>For example:</p> |
| <ul> |
| <li>Create a text file containing the user names and passwords, separated by commas. |
| Put this in the same directory as your test plan. |
| </li> |
| <li> |
| Add a CSV DataSet configuration element to the test plan. |
| Name the variables <code>USER</code> and <code>PASS</code>. |
| </li> |
| <li> |
| Replace the login name with <code>${USER}</code> and the password with <code>${PASS}</code> on the appropriate |
| samplers |
| </li> |
| </ul> |
| <p>The CSV Data Set element will read a new line for each thread. |
| </p> |
| </section> |
| |
| <section name="§-num;.7 Reducing resource requirements" anchor="lean_mean"> |
| <p> |
| Some suggestions on reducing resource usage. |
| </p> |
| <ul> |
| <li>Use non-GUI mode: <code>jmeter -n -t test.jmx -l test.jtl</code></li> |
| <li>Use as few Listeners as possible; if using the <code>-l</code> flag as above they can all be deleted or disabled.</li> |
| <li>Don't use "View Results Tree" or "View Results in Table" listeners during the load test, use them only during scripting phase to debug your scripts.</li> |
| <li>Rather than using lots of similar samplers, |
| use the same sampler in a loop, and use variables (CSV Data Set) to vary the sample. |
| [The Include Controller does not help here, as it adds all the test elements in the file to the test plan.] |
| </li> |
| <li>Don't use functional mode</li> |
| <li>Use CSV output rather than XML</li> |
| <li>Only save the data that you need</li> |
| <li>Use as few Assertions as possible</li> |
| <li>Use the most performing scripting language (see JSR223 section)</li> |
| </ul> |
| <p> |
| If your test needs large amounts of data - particularly if it needs to be randomised - create the test data in a file |
| that can be read with CSV Dataset. This avoids wasting resources at run-time. |
| </p> |
| </section> |
| |
| <section name="§-num;.8 BeanShell server" anchor="beanshell_server"> |
| <p> |
| The BeanShell interpreter has a very useful feature - it can act as a server, |
| which is accessible by telnet or http. |
| </p> |
| <note> |
| There is no security. Anyone who can connect to the port can issue any BeanShell commands. |
| These can provide unrestricted access to the JMeter application and the host. |
| <b>Do not enable the server unless the ports are protected against access, e.g. by a firewall.</b> |
| </note> |
| <p> |
| If you do wish to use the server, define the following in <code>jmeter.properties</code>: |
| </p> |
| <source> |
| beanshell.server.port=9000 |
| beanshell.server.file=../extras/startup.bsh |
| </source> |
| <p> |
| In the above example, the server will be started, and will listen on ports <code>9000</code> and <code>9001</code>. |
| Port <code>9000</code> will be used for http access. Port <code>9001</code> will be used for telnet access. |
| The <code>startup.bsh</code> file will be processed by the server, and can be used to define various functions and set up variables. |
| The startup file defines methods for setting and printing JMeter and system properties. |
| This is what you should see in the JMeter console: |
| </p> |
| <source> |
| Startup script running |
| Startup script completed |
| Httpd started on port: 9000 |
| Session started on port: 9001 |
| </source> |
| <p> |
| There is a sample script (<code>extras/remote.bsh</code>) you can use to test the server. |
| [Have a look at it to see how it works.] |
| <br/> |
| When starting it in the JMeter <code>bin</code> directory |
| (adjust paths as necessary if running from elsewhere) |
| the output should look like: |
| <source> |
| $ java -jar ../lib/bshclient.jar localhost 9000 ../extras/remote.bsh |
| Connecting to BSH server on localhost:9000 |
| Reading responses from server … |
| BeanShell 2.0b5 - by Pat Niemeyer (pat@pat.net) |
| bsh % remote.bsh starting |
| user.home = C:\Documents and Settings\User |
| user.dir = D:\eclipseworkspaces\main\JMeter_trunk\bin |
| log_level.jmeter = INFO |
| log_level.jorphan = INFO |
| Setting property 'EXAMPLE' to '0'. |
| Setting property 'EXAMPLE' to '1'. |
| Setting property 'EXAMPLE' to '2'. |
| Setting property 'EXAMPLE' to '3'. |
| Setting property 'EXAMPLE' to '4'. |
| Setting property 'EXAMPLE' to '5'. |
| Setting property 'EXAMPLE' to '6'. |
| Setting property 'EXAMPLE' to '7'. |
| Setting property 'EXAMPLE' to '8'. |
| Setting property 'EXAMPLE' to '9'. |
| EXAMPLE = 9 |
| remote.bsh ended |
| bsh % … disconnected from server. |
| </source> |
| </p> |
| <p> |
| As a practical example, assume you have a long-running JMeter test running in non-GUI mode, |
| and you want to vary the throughput at various times during the test. |
| The test-plan includes a Constant Throughput Timer which is defined in terms of a property, |
| e.g. <code>${__P(throughput)}</code>. |
| The following BeanShell commands could be used to change the test: |
| </p> |
| <source> |
| printprop("throughput"); |
| curr = Integer.decode(args[0]); // Start value |
| inc = Integer.decode(args[1]); // Increment |
| end = Integer.decode(args[2]); // Final value |
| secs = Integer.decode(args[3]); // Wait between changes |
| while(curr <= end) { |
| setprop("throughput",curr.toString()); // Needs to be a string here |
| Thread.sleep(secs*1000); |
| curr += inc; |
| } |
| printprop("throughput"); |
| </source> |
| <p>The script can be stored in a file (<code>throughput.bsh</code>, say), and sent to the server using <code>bshclient.jar</code>. |
| For example: |
| </p> |
| <source> |
| java -jar ../lib/bshclient.jar localhost 9000 throughput.bsh 70 5 100 60 |
| </source> |
| </section> |
| |
| <section name="§-num;.9 BeanShell scripting" anchor="bsh_scripting"> |
| <subsection name="§-num;.9.1 Overview" anchor="bsh_overview"> |
| <p> |
| Each BeanShell test element has its own copy of the interpreter (for each thread). |
| If the test element is repeatedly called, e.g. within a loop, then the interpreter is retained |
| between invocations unless the "<code>Reset bsh.Interpreter before each call</code>" option is selected. |
| For intensive load testing, it is recommended to use a JSR223 scripting language whose ScriptingEngine implements <code>Compilable</code>, |
| see JSR223 section below for more details. |
| </p> |
| <p> |
| Some long-running tests may cause the interpreter to use lots of memory; if this is the case try using the reset option. |
| </p> |
| <p> |
| You can test BeanShell scripts outside JMeter by using the command-line interpreter: |
| <source> |
| $ java -cp bsh-xxx.jar[;other jars as needed] bsh.Interpreter file.bsh [parameters] |
| </source> |
| or |
| <source> |
| $ java -cp bsh-xxx.jar bsh.Interpreter |
| bsh% source("file.bsh"); |
| bsh% exit(); // or use EOF key (e.g. ^Z or ^D) |
| </source> |
| </p> |
| </subsection> |
| <subsection name="§-num;.9.2 Sharing Variables" anchor="bsh_variables"> |
| <p> |
| Variables can be defined in startup (initialisation) scripts. |
| These will be retained across invocations of the test element, unless the reset option is used. |
| </p> |
| <p> |
| Scripts can also access JMeter variables using the <code>get()</code> and <code>put()</code> methods of the "<code>vars</code>" variable, |
| for example: |
| <source>vars.get("HOST"); |
| vars.put("MSG","Successful");</source> |
| The <code>get()</code> and <code>put()</code> methods only support variables with String values, |
| but there are also <code>getObject()</code> and <code>putObject()</code> methods which can be used for arbitrary objects. |
| JMeter variables are local to a thread, but can be used by all test elements (not just Beanshell). |
| </p> |
| <p> |
| If you need to share variables between threads, then JMeter properties can be used: |
| <source> |
| import org.apache.jmeter.util.JMeterUtils; |
| String value = JMeterUtils.getPropDefault("name",""); |
| JMeterUtils.setProperty("name", "value"); |
| </source> |
| The sample <code>.bshrc</code> files contain sample definitions of <code>getprop()</code> and <code>setprop()</code> methods. |
| </p> |
| <p> |
| Another possible method of sharing variables is to use the "<code>bsh.shared</code>" shared namespace. |
| For example: |
| <source> |
| if (bsh.shared.myObj == void){ |
| // not yet defined, so create it: |
| myObj = new AnyObject(); |
| } |
| bsh.shared.myObj.process(); |
| </source> |
| Rather than creating the object in the test element, it can be created in the startup file |
| defined by the JMeter property "<code>beanshell.init.file</code>". This is only processed once. |
| </p> |
| </subsection> |
| </section> |
| |
| <section name="§-num;.10 Developing script functions in BeanShell, Javascript or Jexl etc." anchor="developing_scripts"> |
| <p> |
| It's quite hard to write and test scripts as functions. |
| However, JMeter has the JSR223, BSF (and BeanShell) samplers which can be used instead. |
| </p> |
| <p> |
| Create a simple Test Plan containing the JSR223 or BSF Sampler and Tree View Listener. |
| Code the script in the sampler script pane, and test it by running the test. |
| If there are any errors, these will show up in the Tree View. |
| Also the result of running the script will show up as the response. |
| </p> |
| <p> |
| Once the script is working properly, it can be stored as a variable on the Test Plan. |
| The script variable can then be used to create the function call. |
| For example, suppose a BeanShell script is stored in the variable <code>RANDOM_NAME</code>. |
| The function call can then be coded as <code>${__BeanShell(${RANDOM_NAME})}</code>. |
| There is no need to escape any commas in the script, |
| because the function call is parsed before the variable's value is interpolated. |
| </p> |
| </section> |
| |
| <section name="§-num;.11 Parameterising tests" anchor="parameterising_tests"> |
| <p> |
| Often it is useful to be able to re-run the same test with different settings. |
| For example, changing the number of threads or loops, or changing a hostname. |
| </p> |
| <p> |
| One way to do this is to define a set of variables on the Test Plan, and then use those variables in the test elements. |
| For example, one could define the variable <code>LOOPS=10</code>, and refer to that in the Thread Group as <code>${LOOPS}</code>. |
| To run the test with 20 loops, just change the value of the <code>LOOPS</code> variable on the Test Plan. |
| </p> |
| <p> |
| This quickly becomes tedious if you want to run lots of tests in non-GUI mode. |
| One solution to this is to define the Test Plan variable in terms of a property, |
| for example <code>LOOPS=${__P(loops,10))}</code>. |
| This uses the value of the property "<code>loops</code>", defaulting to <code>10</code> if the property is not found. |
| The "<code>loops</code>" property can then be defined on the JMeter command-line: |
| <source>jmeter … -Jloops=12 …</source> |
| If there are a lot of properties that need to be changed together, |
| then one way to achieve this is to use a set of property files. |
| The appropriate property file can be passed in to JMeter using the <code>-q</code> command-line option. |
| </p> |
| </section> |
| |
| <section name="§-num;.12 JSR223 Elements" anchor="jsr223"> |
| <p> |
| For intensive load testing, the recommended scripting language is one whose ScriptingEngine implements the <code>Compilable</code> interface. |
| Groovy scripting engine implements <code>Compilable</code>. However neither Beanshell nor Javascript do so as of release date of JMeter 2.13, so it is |
| recommended to avoid them for intensive load testing. |
| <note>Note: Beanshell implements the <code>Compilable</code> interface but it has not been coded - the method just throws an Exception. |
| JMeter has an explicit work-round for this bug.</note> |
| |
| When using JSR 223 elements, always set caching key to a unique value to ensure the script compilation is cached if underlying language supports it. |
| Ensure the script does not use any variable using <code>${varName}</code> as caching would take only first value of <code>${varName}</code>. Instead use : |
| <source> |
| vars.get("varName") |
| </source> |
| </p> |
| <p> |
| You can also pass them as Parameters to the script and use them this way. |
| </p> |
| </section> |
| |
| <section name="§-num;.13 Sharing variables between threads and thread groups" anchor="sharing_variables"> |
| <p> |
| Variables are local to a thread; a variable set in one thread cannot be read in another. |
| This is by design. For variables that can be determined before a test starts, see |
| <a href="#parameterising_tests">Parameterising Tests</a> (above). |
| If the value is not known until the test starts, there are various options: |
| <ul> |
| <li>Store the variable as a property - properties are global to the JMeter instance</li> |
| <li>Write variables to a file and re-read them.</li> |
| <li>Use the <code>bsh.shared</code> namespace - see <a href="#bsh_variables">above</a></li> |
| <li>Write your own Java classes</li> |
| </ul> |
| </p> |
| </section> |
| |
| <section name="§-num;.14 Managing properties" anchor="properties"> |
| <p>When you need to modify jmeter properties, ensure you don't modify <code>jmeter.properties</code> file, |
| <b>instead copy the property from <code>jmeter.properties</code> and modify its value in <code>user.properties</code> file</b>.<br/> |
| Doing so will ease you migration to the next version of JMeter. <br/> |
| Note that in the documentation <code>jmeter.properties</code> is frequently mentioned but this should be understood as |
| "Copy from <code>jmeter.properties</code> to <code>user.properties</code> the property you want to modify and do so in the latter file".</p> |
| <note><code>user.properties</code> file supersedes the properties defined in <code>jmeter.properties</code></note> |
| </section> |
| |
| <section name="§-num;.15 Deprecated elements" anchor="deprecation"> |
| <p>It is advised not to use deprecated elements (marked as such in <a href="../changes.html">changes list</a> and in <a href="./component_reference.html">component reference</a>) |
| and to migrate to new advised elements if available or new way of doing the same thing. <br/> |
| Deprecated elements are removed from the menu in version N but can be enabled for migration by modifying <code>not_in_menu</code> property in <code>user.properties</code> file and removing the full class name |
| of the element from there.<br/></p> |
| <note>Please note that deprecated elements in version N will be removed definitely in version N+1, so ensure you stop using them as soon as possible.</note> |
| </section> |
| |
| </body> |
| </document> |