Merge pull request #353 from mithril-security/readv-fix-pr
fix: correct buffer overwrite in sgx_libc::ocall::{read, pread64, readv, preadv64}diff --git a/Readme.md b/Readme.md
index a44b8f9..1c0f1c3 100644
--- a/Readme.md
+++ b/Readme.md
@@ -2,196 +2,116 @@
[](https://gitter.im/rust-sgx-sdk/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
-# Rust SGX SDK
-Rust SGX SDK helps developers write Intel SGX applications in Rust programming language. [[CCS'17 Paper pdf]](documents/ccsp17.pdf) [[CCS'19 Paper pdf]](https://dingelish.com/ccs19.pdf)
+# Teaclave SGX SDK
+The Teaclave SGX SDK (also known as the Rust SGX SDK) helps developers write Intel SGX applications in the Rust programming language. <sup>[[1]](documents/ccsp17.pdf) [[2]](https://dingelish.com/ccs19.pdf)</sup> Please read our documentation for SDK crates which can be found [here](https://teaclave.apache.org/api-docs/sgx-sdk/sgx_tstd/index.html.) or can be autogenerated with `cargo doc`.
-Good News! Our paper "Building and Maintaining a Third-Party Library Supply Chain for Productive and Secure SGX Enclave Development" has been accepted by ICSE'20 SEIP track. See you guys in Seoul!
+To help users understand our project and know how to use it, we are writing some [wiki](https://teaclave.apache.org/sgx-sdk-docs/) articles. Please contact us on our [mailing list](https://teaclave.apache.org/community/#mailing-lists) if you'd like to see other topics. We'll add it as soon as possible.
-We open-sourced [gbdt-rs](https://github.com/mesalock-linux/gbdt-rs), a light-weight, amazingly fast, memory safe, and trustworthy gradient boosting decision tree implementation and the [paper](documents/gbdt.pdf) has been accepted by IEEE S&P'19! It is optimized for SGX!
-
-To achieve better security, we recommend developers to apply [Non-bypassable Security Paradigm (NbSP)](https://github.com/apache/teaclave-sgx-sdk/blob/master/documents/nbsp.pdf) to the system design and implementation.
-
-To help understand this project and know how to use it, we are writing some [wiki](https://github.com/apache/teaclave-sgx-sdk/wiki) articles. Please [send me an email](mailto:rustsgx@gmail.com?subject=Wiki%20page%20needed:) if you'd like to see other topics. We'll add it as soon as possible.
-
-Current wiki pages:
-
-* [Mitigation of Intel SA 00219 in Rust SGX](https://github.com/apache/incubator-mesatee-sgx/wiki/Mitigation-of-Intel-SA-00219-in-Rust-SGX)
-
-* [`is_x86_feature_detected` in Rust SGX SDK](https://github.com/apache/incubator-mesatee-sgx/wiki/%60is_x86_feature_detected%60-in-Rust-SGX-SDK)
-
-* [The World of Forked crates](https://github.com/apache/teaclave-sgx-sdk/wiki/The-World-of-Forked-crates) introduces the forked crate ecosystem, and provides some guidelines and usage, and show how we secure them.
-
-* [Setup gdb 7.11 on Ubuntu 18.04 for VSCode sgx-gdb remote debugging](https://github.com/apache/teaclave-sgx-sdk/wiki/Setup-gdb-7.11-on-Ubuntu-18.04-for-VSCode---sgx-gdb-remote-debugging) If you encounter errors like `gdb.error: syntax error in expression, near )0x7ffff4127370 = 0.`, probably you need to follow this instruction to setup gdb 7. Thanks to @akoskinas for this great instruction!
-
-* [Performance Optimization Tips](https://github.com/apache/teaclave-sgx-sdk/wiki/Performance-Optimization-Tips)
-
-* [Use VSCode rls+rust-analysis+sgx-gdb for graphic developing (not in docker)](https://github.com/apache/teaclave-sgx-sdk/wiki/Use-VSCode---rls---rust-analysis---sgx-gdb-for-graphic-developing-(not-in-docker))
-
-* [Debugging local enclave in docker](https://github.com/apache/teaclave-sgx-sdk/wiki/Debugging-a-local-Rust-SGX-enclave-in-docker-with-sgx-gdb)
-
-* Everything about [environment setup](https://github.com/apache/teaclave-sgx-sdk/wiki/Environment-Setup)
-
-## v1.1.3
-
-Supports Intel SGX SDK v2.12, and Rust nightly-2020-10-25. Added support to Ubuntu 20.04. **We strongly recommend users to upgrade to Intel SGX SDK v2.12 and drivers to DCAP 1.36.2 and OOT 2.11.0.** [release_notes](release_notes.md).
-
-## v1.1.2
-
-Supports Intel SGX SDK v2.9.1, and Rust nightly-2020-04-07. v1.1.2 provides a handy crate `sgx_signal`, which enables signal capture. One can easily find the place where exception happens and finally triggered `ud2`. And we added `Backtrace::capture` in sgx_tstd. With the help of Intel SGX SDk v2.9.1's patch, dtor of thread local storage finally works on regular SGX thread and pthread thread. Removed sgx_core_futures since Rust is supporting `async`/`await` in `no_std` environment. Please refer to [release_notes](release_notes.md) for more details.
-
-## v1.1.1
-
-Supports Intel SGX SDK v2.9, and Rust nightly-2020-03-12. v1.1.1 contains a bunch of bug fix and new proc macro `sgx_align` to help with aligning given structure. For LVI migigation, it only works on C/C++ parts (EDL headers/Intel's libs) and supports both two modes: `MITIGATION-CVE-2020-0551=LOAD` or `MITIGATION-CVE-2020-0551=CF`. To enable it, one need `env "MITIGATION-CVE-2020-0551=LOAD"` to set this environment variable. For detailed information, please refer to [release_notes](release_notes.md) for more details.
-
-## v1.1.0
-
-Supports Intel SGX SDK v2.7.1, and Rust nightly-2019-11-25. v1.1.0 brings up dynamic static supports by `thread::spawn`, and almost everything of `std::sync`. Also v1.1.0 benefits from Intel SGX SDK's aligned memory allocation primitives to mitigate [INTEL-SA-00219](https://github.com/apache/incubator-mesatee-sgx/wiki/Mitigation-of-Intel-SA-00219-in-Rust-SGX). Besides, we enabled [`is_x86_feature_detected!`](https://github.com/apache/incubator-mesatee-sgx/wiki/%60is_x86_feature_detected%60-in-Rust-SGX-SDK) by parsing a hidden global CPU feature indicator initialized by Intel SGX urts/trts. And we provided Dockerfile for Fedora 27. For detailed information, please refer to [release_notes](release_notes.md) for more details.
-
-## v1.0.9 Release
-
-Supports Intel SGX SDK v2.6, and Rust nightly-2019-08-01. Bumps everything to edition. Removed third_party directory since we have all of those dependencies forked and maintained with merge bot. Since Intel SGX SDK v2.6 imports some breaking changes in global thread metata, thread local features of v1.0.9 is not works on Intel SGX SDK v2.5. EDL and common headers are changed respectively. For detailed information, please refer to [release_notes](release_notes.md) for more details.
-
-## v1.0.8 Release
-
-Supports the most recent Rust nightly (nightly-2019-05-22) and Rust stable (stable-2019-05-14). Code coverage support has been added to sgx_cov. Bug fixes in memory allocator and panicking routines. New third party libraries to support kvdb-memorydb. Please refer to [release_notes](release_notes.md) for more details.
-
-## v1.0.7 Release
-
-Supports Intel SGX SDK v2.5. Master branch supports Rust nightly build (nightly-2019-04-26) and stable branch supports Rust stable build (stable-2019-04-25). Refactored `sgx_tstd` to support `mio`. More sample codes added, including Java/Go clients for ue-ra (Thanks to @bradyjoestar)!. And we are maintaining forks of popular crates on Github organization [mesalock-linux](https://github.com/mesalock-linux). The ported crates are syncing with the original crates with the help of [Pull](https://pull.now.sh) bot and we manually port almost all tests from the original crates to test if the ported crate works well in SGX. Please refer to [release_notes](release_notes.md) for further details.
-
-We changed the built-in EDL files. Please carefully upgrade your EDL files on `import` statements. If you encountered any problems during compilation, please create issue and let me know. Thanks!
-
-**ATTENTION**: (Ubuntu Channel) Starts from Intel SGX SDK 2.8, `aesmd` requires a environment variable to start. If you are using docker, please start `aesmd` as:
-```
-LD_LIBRARY_PATH=/opt/intel/sgx-aesm-service/aesm /opt/intel/sgx-aesm-service/aesm/aesm_service
-```
-
-Starts from Intel SGX SDK 2.5, `aesmd` requires a environment variable to start. If you are using docker, please start `aesmd` as:
-```
-LD_LIBRARY_PATH=/opt/intel/libsgx-enclave-common/aesm /opt/intel/libsgx-enclave-common/aesm/aesm_service
-```
-
-(CentOS Channel) As of 2.6, CentOS branch of Intel SGX SDK is still in format of bin executable. Please start the `aesmd` as past:
-```
-LD_LIBRARY_PATH=/opt/intel/sgxpsw/aesm /opt/intel/sgxpsw/aesm/aesm_service
-```
-
-## v1.0.6 Release
-Fix bugs in sgx_alloc, sgx_types, ucd-generate and improve sgx_tunittest. Added rust-base58. Thanks to @elichai, @cbeck88, @brenzi and @nhynes.
-
-## v1.0.5 Release
-This version supports Rust nightly build (nightly-2019-01-28, v1.34.0) in the master branch and the most recent stable build (stable-2019-01-16, v1.32.0) in the rust-stable branch. It supports the latest Intel SGX SDK **v2.4.0** and Ubuntu Linux 16.04+18.04. We provide support to Intel's Protected Code Loader. We provide sgx_ucrypto and sgx_crypto_helper for using SGX-style crypto primitives in untrusted app and RSA keypair serialization/deserialization in both trusted and untrusted programs. We re-organize ocall related interfaces and provide them in a new crate sgx_libc with a bunch of new ocall functions. In addition, we port net2 to SGX. Please refer to [release_notes](release_notes.md) for further details.
-
-## v1.0.4 Release
-This version supports Rust nightly build (nightly-2018-10-01) in the master branch and the most recent stable build (stable-2018-09-25) in the rust-stable branch. It supports the latest Intel SGX SDK **v2.3.1** and Ubuntu Linux 18.04. It now contains further third party libraries including: bit-vec, chrono, erased-serde, fxhash, nan-preserving-float, num-bigint, quick-error, raft-rs, time, webpki-roots, and yasna. Some third party libraries, like untrusted, parity-wasm and lazy-static, are removed because they support `no_std` and can be used directly from crates.io. We strongly recommend developers upgrade to v1.0.4 and use the most recent Rust release to build it due to the [Security advisory for the standard library](https://blog.rust-lang.org/2018/09/21/Security-advisory-for-std.html). Please refer to [release_notes](release_notes.md) for further details.
-
-## v1.0.1 Release
-This version supports the Rust nightly build (nightly-2018-07-16) in master branch and the most recent Rust stable build (stable-2018-07-10). And it supports the latest Intel SGX SDK **v2.2**. New third party libraries include: bytes, http, iovec, rust-crypto, rust-fnv and rust-threshold-secret-sharing. New code sample 'secretsharing' and 'rust-threshold-secret-sharing' is provided by @davidp94. Please refer to [release_notes](release_notes.md) for further details.
-
-## v1.0.0 Release
-We proudly announce v1.0.0 of rust-sgx-sdk! We port Parity's [Webassembly Interpreter](https://github.com/paritytech/wasmi) to Intel SGX and provide a full functional in-enclave [wasmi sample](samplecode/wasmi), and a [sample solution](samplecode/psi) of two-party private-set-intersection resisting side-channel attacks! From this version, we start to support most recent stable branch of Rust instead of nightly for better stability and future production use. Thus, the [stable branch](https://github.com/apache/teaclave-sgx-sdk/tree/rust-stable) of v1.0.0 supports the most recent Rust stable toolchain (1.26.0 stable-2018-05-07), while the master only supports Rust nightly toolchain of nightly-2018-04-11. Please refer to [release_notes](release_notes.md) for further details.
-
-## v0.9.8 Release
-This version provides security updates regards to recent Spectre attacks in Intel SGX, and supports **Rust stable (2018-03-01)** (in branch named 'rust-stable'). It contains support of [Intel SGX SDK 2.1.2](https://download.01.org/intel-sgx/linux-2.1.2/) and a series of API functions to stop speculative execution on demand. In addition, we provide a ported version of [rust-protobuf](https://crates.io/crates/protobuf) v1.4.4. Please refer to [release_notes](release_notes.md) for further details.
-
-## v0.9.7 Release
-This version provides a new namespace: `sgx_tstd::untrusted`, including `sgx_tstd::untrusted::fs` `sgx_tstd::untrusted::time` and `sgx_tstd::untrusted::path`, providing supports to operation to ocalls in a **untrusted** namespace. The **untrusted** namespace is always enabled no matter `untrusted_*` is set or not. We **urge** the developers to use the `sgx_tstd::untrusted` namespace to port their crates, instead of using the `untrusted_` series of features. Also, we renamed the `untrusted_net` feature to `net` for feature name unification. Please refer to [release_notes](release_notes.md) for further details.
-
-## Run Rust SGX applications in Mesalock Linux
-[MesaLock Linux](https://github.com/mesalock-linux/mesalock-distro) is a general purpose Linux distribution which aims to provide a safe and secure user space environment. Now we can run Rust SGX applications in Mesalock Linux within a few steps. Please refer to the [tutorial](documents/sgx_in_mesalock_linux.md) for details.
-
-## Requirement
-
-Ubuntu 16.04 or 18.04
-
-[Intel SGX SDK 2.9.1 for Linux](https://01.org/intel-software-guard-extensions/downloads) installed
-
-Docker (Recommended)
-
+# Getting Started
+The SDK container can either be built from source or pulled from docker hub and run as a docker container.
## Configuration
The docker image now supports Intel ME. If you need it, please refer to the sgxtime [readme](documents/sgxtime.md) for instructions.
-### Native without docker (Not recommended)
+## Prerequisites
-Install Intel SGX driver and SDK first. And refer to [Dockerfile](dockerfile/Dockerfile.1604.nightly) or stable branch [Dockerfile](Dockerfile.1604.stable) to setup your own native Rust-SGX environment.
+* #### Docker
+* #### Intel SGX OOT 2.11.0 Driver or DCAP 1.36.2 Driver
+* #### Intel SGX SDK v2.12
+* #### Intel SGX PSW
+* #### Rust nightly-2020-10-25
-### Using docker (Recommended) without ME support
+You can find the [installation guides](https://download.01.org/intel-sgx/sgx-linux/2.9/docs/) for Intel SGX software on the 01.org website.
-* As of v1.1.3, we provide 5 docker images: `baiduxlab/sgx-rust:1604-1.1.3` `baiduxlab/sgx-rust:1804-1.1.3` `baiduxlab/sgx-rust:2004-1.1.3` `baiduxlab/sgx-rust:fedora31-1.1.3`, `baiduxlab/sgx-rust:centos8-1.1.3`. The `latest` tag pins on `baiduxlab/sgx-rust:1804-1.1.3`.
+**Note**: if you are running our SDK on a machine without SGX support, you will still need the simulation versions of the Intel PSW and SDK.
-First, make sure Intel SGX Driver for 2.11 is installed and functions well. `/dev/isgx` should appear. If you use DCAP driver, please ensure `/dev/sgx/enclave` and `/dev/sgx/provision` both appear.
+## Pulling a Pre-Built Docker Container
-`$ docker pull baiduxlab/sgx-rust`
+It is assumed that the user has [correctly installed docker](https://docs.docker.com/get-docker/). We provide 5 containers:
-Third, start a docker with sgx device support and the Rust SGX SDK.
+* `baiduxlab/sgx-rust:1604-1.1.3`
+* `baiduxlab/sgx-rust:1804-1.1.3`
+* `baiduxlab/sgx-rust:2004-1.1.3`
+* `baiduxlab/sgx-rust:fedora31-1.1.3`
+* `baiduxlab/sgx-rust:centos8-1.1.3`
-`$ docker run -v /your/path/to/rust-sgx:/root/sgx -ti --device /dev/isgx baiduxlab/sgx-rust`
+With `latest` pinned to `1604-1.1.3`.
-If you use DCAP, the command is:
+First, pull the docker container of your choosing, this command will download `latest`:
+```
+$ docker pull baiduxlab/sgx-rust
+```
+To run the container, we recommend that you download our samplecode
+### Running with Intel SGX Drivers:
+We recommend starting by using the github repository as a first volume to run the container on:
+```
+$ git clone https://github.com/apache/incubator-teaclave-sgx-sdk.git
+```
+To run the container with SGX support, run:
+```
+$ docker run -v /path/to/rust-sgx:/root/sgx --device /dev/libsgx -ti baiduxlab/sgx-rust
+```
+If instead, you are using DCAP drivers you must run:
+```
+$ docker run -v /path/to/rust-sgx:/root/sgx -ti --device /dev/sgx/enclave --device /dev/sgx/provision baiduxlab/sgx-rust
+```
+Whilst inside the container, we must start the AESM daemon and define `LD_LIBRARY_PATH`:
+```
+# LD_LIBRARY_PATH=/opt/intel/sgx-aesm-service/aesm/
-`$ docker run -v /your/path/to/rust-sgx:/root/sgx -ti --device /dev/sgx/enclave --device /dev/sgx/provision baiduxlab/sgx-rust`
+# /opt/intel/sgx-aesm-service/aesm/aesm_service
+```
+If everything has been properly configured, it is now possible to run a quick `helloworld` program inside the container:
+```
+# cd sgx/samplecode/helloworld
-Next, start the aesm service inside the docker
+# make
-`root@docker:/# LD_LIBRARY_PATH=/opt/intel/sgx-aesm-service/aesm/ /opt/intel/sgx-aesm-service/aesm/aesm_service &`
+# cd bin
-Finally, check if the sample code works
+# ./app
+```
+We recommend you look at other files in the `samplecode` folder to familiarize yourself with programming in our SDK.
-`root@docker:~/sgx/samplecode/helloworld# make`
+### Running without Intel SGX Drivers:
-`root@docker:~/sgx/samplecode/helloworld# cd bin`
+**Note**: Intel provides a simulation mode so you can develop on regular machines, by building the enclave app using the libraries sgx_urts_sim, lsgx_uae_service_sim, sgx_trts_sim, sgx_tservice_sim.
-`root@docker:~/sgx/samplecode/helloworld/bin# ./app`
+We recommend starting by using the github repository as a first volume to run the container on:
+```
+$ git clone https://github.com/apache/incubator-teaclave-sgx-sdk.git
+```
+To run the container without SGX support, run:
+```
+$ docker run -v /path/to/rust-sgx:/root/sgx -ti baiduxlab/sgx-rust
+```
+Once inside the container, when running any of the `samplecode` you may either:
+- Modify the Makefile and set `SGX_MODE` to `SW`
+- Run `export SGX_MODE=SW`
+- Run `make` with the build flag `SGX_MODE=SW`
-## Build the docker image by yourself
+We may now run our `helloworld` example:
+
+```
+# cd sgx/samplecode/helloworld
+
+# make SGX_MODE=SW
+
+# cd bin
+
+# ./app
+```
+## Building a Docker Image
Make sure Intel SGX SDK is properly installed and service started on the host OS. Then `cd dockerfile` and run `docker build -t rust-sgx-docker -f Dockerfile.1604.nightly .` to build.
-## Use simulation mode for non SGX-enabled machine (includes macOS)
+# Code Samples
-Intel provides a simulation mode so you can develop on regular machines, by building the enclave app using the libraries `sgx_urts_sim`, `lsgx_uae_service_sim`, `sgx_trts_sim`, `sgx_tservice_sim`.
+We provide eighteen code samples to help developers understand how to write Enclave code in Rust. These samples are located in the `samplecode` directory.
-First, pull the docker image. If you'd like to work on stable branch of Rust and `rust-stable` branch of this SDK, please pull `baiduxlab/sgx-rust-stable` instead.
-
-`$ docker pull baiduxlab/sgx-rust`
-
-Second, start a docker with the Rust SGX SDK.
-
-`$ docker run -v /your/path/to/rust-sgx:/root/sgx -ti baiduxlab/sgx-rust`
-
-But when building any sample code, set the `SGX_MODE` to `SW` in `Makefile`.
-
-`root@docker:~/sgx/samplecode/helloworld# vi Makefile`
-
-Replace `SGX_MODE ?= HW` with `SGX_MODE ?= SW`
-
-or run `export SGX_MODE=SW` in your terminal.
-
-Finally, check if the sample code works
-
-`root@docker:~/sgx/samplecode/helloworld# make`
-
-`root@docker:~/sgx/samplecode/helloworld# cd bin`
-
-`root@docker:~/sgx/samplecode/helloworld/bin# ./app`
-
-If not set, you could get an error:
-```
-Info: Please make sure SGX module is enabled in the BIOS, and install SGX driver afterwards.
-Error: Invalid SGX device.
-```
-
-# Documents
-
-The online documents for SDK crates can be found [here](https://dingelish.github.io/).
-
-We recommend to use `cargo doc` to generate documents for each crate in this SDK by yourself. The auto generated documents are easy to read and search.
-
-# Sample Codes
-
-We provide eighteen sample codes to help developers understand how to write Enclave codes in Rust. These codes are located at `samplecode` directory.
+<details>
+<summary>See Code Samples</summary>
* `helloworld` is a very simple app. It shows some basic usages of argument passing, Rust string and ECALL/OCALLs.
@@ -240,9 +160,13 @@
* `sgx-cov` shows how to use lcov with Rust SGX enclave to generate code coverage report. See the [readme](samplecode/sgx-cov/Readme.md) for details.
* `tcmalloc` shows how to link Rust-SGX enclave with tcmalloc (provided by Intel SGX SDK), and test its performance with different kinds of workload.
+</details>
# Tips for writing enclaves in Rust
+<details>
+<summary>See tips</summary>
+
## Writing EDL
* For fixed-length array in ECALL/OCALL definition, declare it as an array. For dynamic-length array, use the keyword `size=` to let the Intel SGX knows how many bytes should be copied.
@@ -257,6 +181,130 @@
* For Fixed-length array, the above method is acceptable. And according to discussions in [issue 30382](https://github.com/rust-lang/rust/issues/30382) and [issue 31227](https://github.com/rust-lang/rust/issues/31227), thin-pointers (such as fixed-length array) are FFI-safe for now, but undocumented. In the sample codes, we use fixed-length arrays for passing and returning some fixed-length data.
+## Non-bypassable Security Paradigm (NbSP)
+
+* To achieve better security, we recommend developers to apply [Non-bypassable Security Paradigm (NbSP)](https://github.com/apache/teaclave-sgx-sdk/blob/master/documents/nbsp.pdf) to the system design and implementation.
+</details>
+
+# Releases
+## Latest: v1.1.3
+
+Supports Intel SGX SDK v2.12, and Rust nightly-2020-10-25. Added support to Ubuntu 20.04. **We strongly recommend users to upgrade to Intel SGX SDK v2.12 and drivers to DCAP 1.36.2 and OOT 2.11.0.** [release_notes](release_notes.md).
+
+
+<details>
+<summary>Version 1.1.2</summary>
+
+## v1.1.2
+
+Supports Intel SGX SDK v2.9.1, and Rust nightly-2020-04-07. v1.1.2 provides a handy crate `sgx_signal`, which enables signal capture. One can easily find the place where exception happens and finally triggered `ud2`. And we added `Backtrace::capture` in sgx_tstd. With the help of Intel SGX SDk v2.9.1's patch, dtor of thread local storage finally works on regular SGX thread and pthread thread. Removed sgx_core_futures since Rust is supporting `async`/`await` in `no_std` environment. Please refer to [release_notes](release_notes.md) for more details.
+</details>
+
+<details>
+<summary>Version 1.1.1</summary>
+
+## v1.1.1
+
+Supports Intel SGX SDK v2.9, and Rust nightly-2020-03-12. v1.1.1 contains a bunch of bug fix and new proc macro `sgx_align` to help with aligning given structure. For LVI migigation, it only works on C/C++ parts (EDL headers/Intel's libs) and supports both two modes: `MITIGATION-CVE-2020-0551=LOAD` or `MITIGATION-CVE-2020-0551=CF`. To enable it, one need `env "MITIGATION-CVE-2020-0551=LOAD"` to set this environment variable. For detailed information, please refer to [release_notes](release_notes.md) for more details.
+</details>
+
+<details>
+<summary>Version 1.1.0</summary>
+
+## v1.1.0
+
+Supports Intel SGX SDK v2.7.1, and Rust nightly-2019-11-25. v1.1.0 brings up dynamic static supports by `thread::spawn`, and almost everything of `std::sync`. Also v1.1.0 benefits from Intel SGX SDK's aligned memory allocation primitives to mitigate [INTEL-SA-00219](https://github.com/apache/incubator-mesatee-sgx/wiki/Mitigation-of-Intel-SA-00219-in-Rust-SGX). Besides, we enabled [`is_x86_feature_detected!`](https://github.com/apache/incubator-mesatee-sgx/wiki/%60is_x86_feature_detected%60-in-Rust-SGX-SDK) by parsing a hidden global CPU feature indicator initialized by Intel SGX urts/trts. And we provided Dockerfile for Fedora 27. For detailed information, please refer to [release_notes](release_notes.md) for more details.
+</details>
+
+<details>
+<summary>Version 1.0.9</summary>
+
+## v1.0.9 Release
+
+Supports Intel SGX SDK v2.6, and Rust nightly-2019-08-01. Bumps everything to edition. Removed third_party directory since we have all of those dependencies forked and maintained with merge bot. Since Intel SGX SDK v2.6 imports some breaking changes in global thread metata, thread local features of v1.0.9 is not works on Intel SGX SDK v2.5. EDL and common headers are changed respectively. For detailed information, please refer to [release_notes](release_notes.md) for more details.
+</details>
+
+<details>
+<summary>Version 1.0.8</summary>
+
+## v1.0.8 Release
+
+Supports the most recent Rust nightly (nightly-2019-05-22) and Rust stable (stable-2019-05-14). Code coverage support has been added to sgx_cov. Bug fixes in memory allocator and panicking routines. New third party libraries to support kvdb-memorydb. Please refer to [release_notes](release_notes.md) for more details.
+</details>
+
+<details>
+<summary>Version 1.0.7</summary>
+
+## v1.0.7 Release
+
+Supports Intel SGX SDK v2.5. Master branch supports Rust nightly build (nightly-2019-04-26) and stable branch supports Rust stable build (stable-2019-04-25). Refactored `sgx_tstd` to support `mio`. More sample codes added, including Java/Go clients for ue-ra (Thanks to @bradyjoestar)!. And we are maintaining forks of popular crates on Github organization [mesalock-linux](https://github.com/mesalock-linux). The ported crates are syncing with the original crates with the help of [Pull](https://pull.now.sh) bot and we manually port almost all tests from the original crates to test if the ported crate works well in SGX. Please refer to [release_notes](release_notes.md) for further details.
+
+We changed the built-in EDL files. Please carefully upgrade your EDL files on `import` statements. If you encountered any problems during compilation, please create issue and let me know. Thanks!
+
+**ATTENTION**: (Ubuntu Channel) Starts from Intel SGX SDK 2.8, `aesmd` requires a environment variable to start. If you are using docker, please start `aesmd` as:
+```
+LD_LIBRARY_PATH=/opt/intel/sgx-aesm-service/aesm /opt/intel/sgx-aesm-service/aesm/aesm_service
+```
+
+Starts from Intel SGX SDK 2.5, `aesmd` requires a environment variable to start. If you are using docker, please start `aesmd` as:
+```
+LD_LIBRARY_PATH=/opt/intel/libsgx-enclave-common/aesm /opt/intel/libsgx-enclave-common/aesm/aesm_service
+```
+
+(CentOS Channel) As of 2.6, CentOS branch of Intel SGX SDK is still in format of bin executable. Please start the `aesmd` as past:
+```
+LD_LIBRARY_PATH=/opt/intel/sgxpsw/aesm /opt/intel/sgxpsw/aesm/aesm_service
+```
+</details>
+
+<details>
+<summary>Version 1.0.6</summary>
+
+## v1.0.6 Release
+Fix bugs in sgx_alloc, sgx_types, ucd-generate and improve sgx_tunittest. Added rust-base58. Thanks to @elichai, @cbeck88, @brenzi and @nhynes.
+</details>
+
+<details>
+<summary>Version 1.0.5</summary>
+
+## v1.0.5 Release
+This version supports Rust nightly build (nightly-2019-01-28, v1.34.0) in the master branch and the most recent stable build (stable-2019-01-16, v1.32.0) in the rust-stable branch. It supports the latest Intel SGX SDK **v2.4.0** and Ubuntu Linux 16.04+18.04. We provide support to Intel's Protected Code Loader. We provide sgx_ucrypto and sgx_crypto_helper for using SGX-style crypto primitives in untrusted app and RSA keypair serialization/deserialization in both trusted and untrusted programs. We re-organize ocall related interfaces and provide them in a new crate sgx_libc with a bunch of new ocall functions. In addition, we port net2 to SGX. Please refer to [release_notes](release_notes.md) for further details.
+</details>
+<details>
+<summary>Version 1.0.4</summary>
+
+## v1.0.4 Release
+This version supports Rust nightly build (nightly-2018-10-01) in the master branch and the most recent stable build (stable-2018-09-25) in the rust-stable branch. It supports the latest Intel SGX SDK **v2.3.1** and Ubuntu Linux 18.04. It now contains further third party libraries including: bit-vec, chrono, erased-serde, fxhash, nan-preserving-float, num-bigint, quick-error, raft-rs, time, webpki-roots, and yasna. Some third party libraries, like untrusted, parity-wasm and lazy-static, are removed because they support `no_std` and can be used directly from crates.io. We strongly recommend developers upgrade to v1.0.4 and use the most recent Rust release to build it due to the [Security advisory for the standard library](https://blog.rust-lang.org/2018/09/21/Security-advisory-for-std.html). Please refer to [release_notes](release_notes.md) for further details.
+</details>
+
+<details>
+<summary>Version 1.0.1</summary>
+
+## v1.0.1 Release
+This version supports the Rust nightly build (nightly-2018-07-16) in master branch and the most recent Rust stable build (stable-2018-07-10). And it supports the latest Intel SGX SDK **v2.2**. New third party libraries include: bytes, http, iovec, rust-crypto, rust-fnv and rust-threshold-secret-sharing. New code sample 'secretsharing' and 'rust-threshold-secret-sharing' is provided by @davidp94. Please refer to [release_notes](release_notes.md) for further details.
+</details>
+
+<details>
+<summary>Version 1.0.0</summary>
+
+## v1.0.0 Release
+We proudly announce v1.0.0 of rust-sgx-sdk! We port Parity's [Webassembly Interpreter](https://github.com/paritytech/wasmi) to Intel SGX and provide a full functional in-enclave [wasmi sample](samplecode/wasmi), and a [sample solution](samplecode/psi) of two-party private-set-intersection resisting side-channel attacks! From this version, we start to support most recent stable branch of Rust instead of nightly for better stability and future production use. Thus, the [stable branch](https://github.com/apache/teaclave-sgx-sdk/tree/rust-stable) of v1.0.0 supports the most recent Rust stable toolchain (1.26.0 stable-2018-05-07), while the master only supports Rust nightly toolchain of nightly-2018-04-11. Please refer to [release_notes](release_notes.md) for further details.
+</details>
+
+<details>
+<summary>Version 0.9.8</summary>
+
+## v0.9.8 Release
+This version provides security updates regards to recent Spectre attacks in Intel SGX, and supports **Rust stable (2018-03-01)** (in branch named 'rust-stable'). It contains support of [Intel SGX SDK 2.1.2](https://download.01.org/intel-sgx/linux-2.1.2/) and a series of API functions to stop speculative execution on demand. In addition, we provide a ported version of [rust-protobuf](https://crates.io/crates/protobuf) v1.4.4. Please refer to [release_notes](release_notes.md) for further details.
+</details>
+
+<details>
+<summary>Version 0.9.7</summary>
+
+## v0.9.7 Release
+This version provides a new namespace: `sgx_tstd::untrusted`, including `sgx_tstd::untrusted::fs` `sgx_tstd::untrusted::time` and `sgx_tstd::untrusted::path`, providing supports to operation to ocalls in a **untrusted** namespace. The **untrusted** namespace is always enabled no matter `untrusted_*` is set or not. We **urge** the developers to use the `sgx_tstd::untrusted` namespace to port their crates, instead of using the `untrusted_` series of features. Also, we renamed the `untrusted_net` feature to `net` for feature name unification. Please refer to [release_notes](release_notes.md) for further details.
+</details>
+
# License
The Apache Teaclave Rust-SGX SDK is provided under the Apache license. Please refer to the [License file](LICENSE) for details.
@@ -265,11 +313,6 @@
Ran Duan, Long Li, Chan Zhao, Shi Jia, Yu Ding, Yulong Zhang, Huibo Wang, Yueqiang Cheng, Lenx Wei, Tanghui Chen
-# Acknowledgement
-
-Thanks to [Prof. Jingqiang Lin](http://people.ucas.ac.cn/~0010268) for his contribution to this project.
-
# Contacts
Yu Ding, dingelish@gmail.com
-
diff --git a/documents/README.md b/documents/README.md
index 719704f..3975c5c 100644
--- a/documents/README.md
+++ b/documents/README.md
@@ -4,9 +4,33 @@
# Teaclave SGX SDK Documentation
-* [`is_x86_feature_detected` in Teaclave SGX SDK](/documents/is_x86_feature_detected-in-sgx-sdk)
-* [Developing with Visual Studio Code](/documents/developing-with-vscode/)
+Welcome to the Teaclave Rust-SGX SDK[[1]](documents/ccsp17.pdf) [[2]](https://dingelish.com/ccs19.pdf) documentation wiki. To help users understand our project and know how to use it, we are writing some articles. Please join our [mailing list](https://teaclave.apache.org/community/#mailing-lists) if you'd like to see other topics. We'll add it as soon as possible.
+
+Please read our documentation for SDK crates which can be found [here]( https://teaclave.apache.org/api-docs/sgx-sdk/sgx_tstd/index.html) or can be autogenerated with `cargo doc`.
+
+## Development
+
+* [Setting up your Development Environment](/documents/environment-setup.md)
+* [Rust SGX Applications in Mesalock Linux](/documents/sgx_in_mesalock_linux.md)
+* [`is_x86_feature_detected` in Teaclave SGX SDK](/documents/is_x86_feature_detected-in-sgx-sdk.md)
+* [Developing with Visual Studio Code](/documents/developing-with-vscode.md)
+* [Debugging a Local Enclave in Docker](/documents/debugging-a-local-rust-sgx-enclave-in-docker-with-sgx-gdb.md)
+* [Public Cloud Rust SGX Development](/documents/public-cloud-for-rust-sgx-development.md)
+* [The World of Forked Crates](/documents/the-world-of-forked-crates.md)
+
+## Tutorials
+
+* [Acquiring Trusted Timestamp from Intel ME in SGX enclave](/documents/sgxtime.md)
+* [Performance Optimization Tips](/documents/performance-optimization-tops.md)
+* [Using `gdb` in Ubuntu 18 VSCode for Remote Debugging](/documents/setup-gdb-7.11-on-ubuntu-18.04-for-vscode---sgx-remote-debugging.md)
+* [Using `gdb` in VSCode for Graphic Developing (Not in Docker)](/documents/use-vscode---rls---rust-analysis---sgx-gdb-for-graphic-developing-(not-in-docker))
## Security
-* [Everything about CVE-2020-5499](/documents/everything-about-cve-2020-5499/)
+* [Everything about CVE-2020-5499](/documents/everything-about-cve-2020-5499.md)
+* [Non-Bypassable Security Protocol](/documents/nbsp.pdf)
+* [Mitigation of Intel SA 00219 in Rust-SGX](/documents/mitigation-of-intel-sa-00219-in-rust-sgx.md)
+
+## Applications
+
+* [Fast and Trustworthy Gradient Boosting Decision Tree](/documents/gbdt.pdf)
diff --git a/documents/ccsp17.pdf b/documents/ccsp17.pdf
deleted file mode 100644
index 9bb68c3..0000000
--- a/documents/ccsp17.pdf
+++ /dev/null
Binary files differ
diff --git a/documents/debugging-a-local-rust-sgx-enclave-in-docker-with-sgx-gdb.md b/documents/debugging-a-local-rust-sgx-enclave-in-docker-with-sgx-gdb.md
new file mode 100644
index 0000000..fdf35e7
--- /dev/null
+++ b/documents/debugging-a-local-rust-sgx-enclave-in-docker-with-sgx-gdb.md
@@ -0,0 +1,236 @@
+---
+permalink: /sgx-sdk-docs/debugging-a-local-rust-sgx-enclave
+---
+# Before start
+
+As of today (03/19/2019), `sgx-gdb` cannot work well with gdb > 7.12. In this wiki page, I'm using the gdb 7.11.1. Please check if you have the correct version of gdb.
+
+# Requirement
+
+* Linux
+* docker installed
+* Intel SGX driver installed and `/dev/isgx` works.
+* docker image baiduxlab/sgx-rust:1604
+
+# Prepare the code
+
+Let's use [hello-rust](https://github.com/apache/incubator-teaclave-sgx-sdk/tree/master/samplecode/hello-rust) as debuggee. We need to add debug info for all of the Rust/C codes.
+
+First, switch to debug build for the Rust codes. In the root Makefile of hello-rust, remove the `--release` Rust flag and fix the path:
+```diff
+-App_Rust_Flags := --release
++App_Rust_Flags :=
+ App_SRC_Files := $(shell find app/ -type f -name '*.rs') $(shell find app/ -type f -name 'Cargo.toml')
+ App_Include_Paths := -I ./app -I./include -I$(SGX_SDK)/include -I$(CUSTOM_EDL_PATH)
+ App_C_Flags := $(SGX_COMMON_CFLAGS) -fPIC -Wno-attributes $(App_Include_Paths)
+
+-App_Rust_Path := ./app/target/release
++App_Rust_Path := ./app/target/debug
+ App_Enclave_u_Object :=app/libEnclave_u.a
+ App_Name := bin/app
+```
+And do the same thing in enclave/Makefile:
+```diff
+- RUSTC_BOOTSTRAP=1 cargo build --release
+- cp ./target/release/libhelloworldsampleenclave.a ../lib/libenclave.a
+-endif
++ RUSTC_BOOTSTRAP=1 cargo build
++ cp ./target/debug/libhelloworldsampleenclave.a ../lib/libenclave.a
++endif
+```
+
+And we also need to add debug symbol to for Enclave_t.c and Enclave_u.c. In the root Makefile:
+```diff
+ SGX_ENCLAVE_SIGNER := $(SGX_SDK)/bin/x86/sgx_sign
+ SGX_EDGER8R := $(SGX_SDK)/bin/x86/sgx_edger8r
+ else
+- SGX_COMMON_CFLAGS := -m64
++ SGX_COMMON_CFLAGS := -m64 -ggdb
+ SGX_LIBRARY_PATH := $(SGX_SDK)/lib64
+ SGX_ENCLAVE_SIGNER := $(SGX_SDK)/bin/x64/sgx_sign
+ SGX_EDGER8R := $(SGX_SDK)/bin/x64/sgx_edger8r
+```
+
+# Start the docker container
+
+Next, use the following command to start a docker container. Please **fix the path** before running it.
+
+```bash
+$ docker run -ti \
+ --rm \
+ --privileged \
+ -v /home/ding/rust-sgx-sdk:/root/rust-sgx-sdk \
+ --device /dev/isgx \
+ baiduxlab/sgx-rust:1604 bash
+root@ef40bc98b273:~#
+```
+
+Then, set up gdb using `apt-get`:
+
+```
+root@ef40bc98b273:~# apt-get update && apt-get install -y gdb
+```
+
+Then check if the version is correct:
+
+```
+root@ef40bc98b273:~# gdb --version
+GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
+Copyright (C) 2016 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law. Type "show copying"
+and "show warranty" for details.
+This GDB was configured as "x86_64-linux-gnu".
+Type "show configuration" for configuration details.
+For bug reporting instructions, please see:
+<http://www.gnu.org/software/gdb/bugs/>.
+Find the GDB manual and other documentation resources online at:
+<http://www.gnu.org/software/gdb/documentation/>.
+For help, type "help".
+Type "apropos word" to search for commands related to "word".
+```
+
+As of today (03-19-2019), the installed version is 7.11.1 and it's ok.
+
+Then we need to start the aesm service daemon:
+
+```
+root@ef40bc98b273:~# /opt/intel/libsgx-enclave-common/aesm/aesm_service
+aesm_service[878]: The server sock is 0x55ed65a9a560
+aesm_service[878]: [ADMIN]White List update requested
+aesm_service[878]: [ADMIN]Platform Services initializing
+aesm_service[878]: [ADMIN]Platform Services initialization failed due to DAL error
+aesm_service[878]: [ADMIN]White list update request successful for Version: 49
+```
+
+Just ignore the `DAL error`.
+
+# Debug the enclave
+
+```
+root@ef40bc98b273:~# cd rust-sgx-sdk/samplecode/hello-rust/
+root@ef40bc98b273:~/rust-sgx-sdk/samplecode/hello-rust# make
+info: syncing channel updates for 'stable-2019-01-17-x86_64-unknown-linux-gnu'
+info: latest update on 2019-01-17, rust version 1.32.0 (9fda7c223 2019-01-16)
+info: downloading component 'rustc'
+..........(suppressed output)..........
+LINK => enclave/enclave.so
+<!-- Please refer to User's Guide for the explanation of each field -->
+<EnclaveConfiguration>
+ <ProdID>0</ProdID>
+ <ISVSVN>0</ISVSVN>
+ <StackMaxSize>0x40000</StackMaxSize>
+ <HeapMaxSize>0x100000</HeapMaxSize>
+ <TCSNum>1</TCSNum>
+ <TCSPolicy>1</TCSPolicy>
+ <DisableDebug>0</DisableDebug>
+ <MiscSelect>0</MiscSelect>
+ <MiscMask>0xFFFFFFFF</MiscMask>
+</EnclaveConfiguration>
+tcs_num 1, tcs_max_num 1, tcs_min_pool 1
+The required memory is 1798144B.
+Succeed.
+SIGN => bin/enclave.signed.so
+```
+
+Let's debug it!
+
+```
+root@ef40bc98b273:~/rust-sgx-sdk/samplecode/hello-rust# cd bin/
+root@ef40bc98b273:~/rust-sgx-sdk/samplecode/hello-rust/bin# sgx-gdb ./app
+GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
+Copyright (C) 2016 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law. Type "show copying"
+and "show warranty" for details.
+This GDB was configured as "x86_64-linux-gnu".
+Type "show configuration" for configuration details.
+For bug reporting instructions, please see:
+<http://www.gnu.org/software/gdb/bugs/>.
+Find the GDB manual and other documentation resources online at:
+<http://www.gnu.org/software/gdb/documentation/>.
+For help, type "help".
+Type "apropos word" to search for commands related to "word"...
+Source directories searched: /opt/sgxsdk/lib64/gdb-sgx-plugin:$cdir:$cwd
+Setting environment variable "LD_PRELOAD" to null value.
+Reading symbols from ./app...done.
+warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
+of file /root/rust-sgx-sdk/samplecode/hello-rust/bin/app.
+Use `info auto-load python-scripts [REGEXP]' to list them.
+(gdb) b say_something
+Breakpoint 1 at 0x11800: file app/Enclave_u.c, line 731.
+(gdb) r
+Starting program: /root/rust-sgx-sdk/samplecode/hello-rust/bin/app
+detect urts is loaded, initializing
+[Thread debugging using libthread_db enabled]
+Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
+[+] Home dir is /root
+[-] Open token file /root/enclave.token error! Will create one.
+add-symbol-file '/root/rust-sgx-sdk/samplecode/hello-rust/bin/enclave.signed.so' 0x7ffff5805340 -readnow -s .interp 0x7ffff5800270 -s .note.gnu.build-id 0x7ffff580028c -s .gnu.hash 0x7ffff58002b0 -s .dynsym 0x7ffff58002e0 -s .dynstr 0x7ffff5800388 -s .gnu.version 0x7ffff58003c2 -s .gnu.version_d 0x7ffff58003d0 -s .rela.dyn 0x7ffff5800408 -s .plt 0x7ffff5805310 -s .plt.got 0x7ffff5805320 -s .nipx 0x7ffff5845060 -s .rodata 0x7ffff58458a0 -s .eh_frame_hdr 0x7ffff584ace0 -s .eh_frame 0x7ffff584d4b0 -s .gcc_except_table 0x7ffff5857850 -s .tbss 0x7ffff5a597a0 -s .init_array 0x7ffff5a597a0 -s .fini_array 0x7ffff5a597a8 -s .data.rel.ro 0x7ffff5a59800 -s .dynamic 0x7ffff5a5b000 -s .got 0x7ffff5a5b190 -s .got.plt 0x7ffff5a5c000 -s .data 0x7ffff5a5c020 -s .nipd 0x7ffff5a5cd84 -s .niprod 0x7ffff5a5cdc0 -s .bss 0x7ffff5a5d600
+[+] Saved updated launch token!
+[+] Init Enclave Successful 2!
+
+Breakpoint 1, say_something (eid=2, retval=0x7fffffffe288,
+ some_string=0x5555557c0f00 "This is a normal world string passed into Enclave!\n", len=51) at app/Enclave_u.c:731
+731 {
+(gdb)
+```
+
+Look at the automatically triggered `add-symbol-file` command. `sgx-gdb` helps us load the debug sym automatically. If you can't see this line, it means that `sgx-gdb` is not working.
+
+Let's check where it stopped at:
+
+```
+(gdb) info reg rip
+rip 0x555555565800 0x555555565800 <say_something>
+```
+
+It stopped at 0x555555565800, not in the enclave. It the place where `say_something` located in `Enclave_u.c`. But we can set another breakpoint at the one inside enclave:
+
+```
+(gdb) info line say_something
+Line 731 of "app/Enclave_u.c" starts at address 0x555555565800 <say_something>
+ and ends at 0x555555565804 <say_something+4>.
+Line 51 of "src/lib.rs" starts at address 0x7ffff5827ab0 <say_something>
+ and ends at 0x7ffff5827ac1 <say_something+17>.
+(gdb) b "src/lib.rs:say_something"
+Note: breakpoint 1 also set at pc 0x7ffff5827ac1.
+Breakpoint 2 at 0x7ffff5827ac1: file src/lib.rs, line 52.
+```
+
+So now bp #2 is the correct bp inside SGX enclave. Continue:
+
+```
+(gdb) c
+Continuing.
+
+Breakpoint 1, say_something (
+ some_string=0x7ffff5a71040 "This is a normal world string passed into Enclave!\n", some_len=51) at src/lib.rs:52
+(gdb) info reg rip
+rip 0x7ffff5827ac1 0x7ffff5827ac1 <say_something+17>
+```
+
+That's it! It stopped correctly at the first statement in enclave function `say_something`! We can do something more using gdb:
+
+```
+(gdb) n
+54 let str_slice = unsafe { slice::from_raw_parts(some_string, some_len) };
+(gdb)
+55 let _ = io::stdout().write(str_slice);
+(gdb)
+This is a normal world string passed into Enclave!
+58 let rust_raw_string = "This is a in-Enclave ";
+(gdb) n
+60 let word:[u8;4] = [82, 117, 115, 116];
+(gdb) p rust_raw_string
+$1 = {data_ptr = 0x7ffff5847150 "This is a in-Enclave Invalid UTF-8\n",
+ length = 21}
+```
+
+That's it! Try `p` more stuffs!
+
+## Tips
+
+[peda](https://github.com/longld/peda) is helpful with some commands like `vmmap`.
diff --git a/documents/developing-with-vscode.md b/documents/developing-with-vscode.md
index 0aa6760..1bc71d2 100644
--- a/documents/developing-with-vscode.md
+++ b/documents/developing-with-vscode.md
@@ -34,7 +34,7 @@
### Create an rls-friendly Teaclave SGX SDK project.
-[hello-rust-vscode-debug](https://github.com/baidu/rust-sgx-sdk/tree/master/samplecode/hello-rust-vscode-debug)
+[hello-rust-vscode-debug](https://github.com/apacher/incubator-teaclave-sgx-sdk/tree/master/samplecode/hello-rust-vscode-debug)
is an example. Differences between this and `hello-rust` are:
1. An extra `Cargo.toml` at the root, which contains two Rust crates: `app` and `enclave`. This change would result in changing the path of `target` folders.
2. Tweak `Makefile` and `enclave/Makefile` and correct the path of `target` folders.
diff --git a/documents/environment-setup.md b/documents/environment-setup.md
new file mode 100644
index 0000000..87e3277
--- /dev/null
+++ b/documents/environment-setup.md
@@ -0,0 +1,139 @@
+---
+permalink: /sgx-sdk-docs/environment-setup
+---
+# Recommended OS to start with
+
+We recommend Ubuntu 16.04/18.04. Desktop or server is the same. It could be your host OS or guest OS (inside docker). Technically, a full compatible list could be found at Intel's download [page](https://download.01.org/intel-sgx/linux-2.4/). As of 04-01-2019 (v 2.4.0), the list contains:
+
+* CentOS 7.5
+* Fedora 27 server
+* RedHat Enterprise Linux 7.4
+* SUSE 12.3 server
+* Ubuntu 16.04
+* Ubuntu 18.04
+
+# Hardware setup
+
+A good reference for hardware compatibility is [SGX-Hardware](https://github.com/ayeks/SGX-hardware). You can use the script [test-sgx.c](https://github.com/ayeks/SGX-hardware/blob/master/test-sgx.c) there to check if SGX is/could be enabled.
+
+Followings are FAQs I've been always asked:
+1. Macbook Pro? No to all on hardware support! Docker-based simulation is OK.
+2. Rack Server? Here are my listings:
+* SuperServer [5019S-MR](https://www.supermicro.com/products/system/1U/5019/SYS-5019S-MR.cfm)
+* Lenovo [SR250](https://www.lenovo.com/us/en/data-center/servers/racks/ThinkSystem-SR250/p/77XX7SRSR25)
+* SuperMicro Microblade server with [MBI-6119G-T4](https://www.supermicro.com/products/MicroBlade/module/MBI-6119G-T4.cfm).
+3. Laptops? I don't know. If you can find something related to Intel SGX in the BIOS, then Yes.
+
+Another solution is Intel's [VCA 2](https://www.intel.com/content/dam/support/us/en/documents/server-products/server-accessories/VCA2_HW_User_Guide.pdf) card. It should be placed only in 2-socket Xeon E5 systems (or following). Dmitrii of Intel Lab is using it for [Redis-SGX](https://redislabs.com/rlsessions_post_type/redis-sgx-secure-redis-intel-sgx/).
+
+# Software Setup
+
+## Rust toolchain
+
+Please use [rustup](https://rustup.rs/) to install and manage Rust toolchains. **DO NOT** use anything like `apt` or `yum`.
+
+During the installation you'll be asked about 'installation options' as follows:
+```
+Current installation options:
+
+ default host triple: x86_64-unknown-linux-gnu
+ default toolchain: stable
+ modify PATH variable: yes
+```
+
+The host triple (though quadruple here) is correct. You could just press enter to skip it. When asking about default toolchain, you could enter `nightly-2019-01-28` or similar version number. And we recommend to answer 'Y' to the PATH modification.
+
+rustup is always installed in `~` and does not affect other users.
+
+Then you will have rustup works well. To switch to another toolchain, try
+```
+$ rustup toolchain default nightly-2019-03-31
+```
+This would triggers downloading and installation if the desired toolchain is not found on your disk.
+
+To add more rust tools such as `rust-src` (for xargo), `rust-clippy` (for lint):
+```
+$ rustup component add rust-src
+```
+
+## Intel SGX toolchain setup
+
+The toolchain setup strictly follows the following steps:
+
+0. Driver installation ( sgx_linux_x64_driver_??????.bin ). You'll get a misc device '/dev/isgx' after this step.
+1. (OPTIONAL, if Intel ME is required) iCls setup (iclsClient-1.45.449.12-1.x86_64.rpm)
+2. (OPTIONAL, if Intel ME is required) jhi setup https://github.com/01org/dynamic-application-loader-host-interface
+3. Platform Software installation (libsgx-enclave-common, libsgx-enclave-common-dev, libsgx-enclave-common-dbgsym)
+4. Intel SGX SDK installation ( sgx_linux_x64_sdk_???????.bin )
+
+And don't forget to source the `environment` file for Intel SGX SDK (such as sgx-sign).
+
+## Docker setup
+
+### Use docker with hardware support, and run aesm inside docker
+
+Firstly, do step 0 to get `/dev/isgx` works. Then start a docker container as follows:
+
+```
+$ docker run -ti --rm -v /path/to/sdk:/root/sgx \
+ --device /dev/isgx \
+ --device /dev/mei0 \ # Optional if you have it and want to use it
+ baiduxlab/sgx-rust
+root@913e6a00c8d8:~#
+```
+
+(Optional) Install iCls and jhi daemon. Steps are [here](https://github.com/apache/incubator-teaclave-sgx-sdk/blob/master/dockerfile/Dockerfile.1604.nightly#L50)
+
+(Optional) Start jhi daemon: `jhid -d`
+
+Start aesm daemon
+```
+root@913e6a00c8d8:~# aesm_service[18]: The server sock is 0x5636e90be960
+aesm_service[18]: [ADMIN]White List update requested
+aesm_service[18]: [ADMIN]Platform Services initializing
+aesm_service[18]: [ADMIN]Platform Services initialization failed due to DAL error
+aesm_service[18]: [ADMIN]White list update request successful for Version: 49
+
+root@913e6a00c8d8:~#
+```
+
+And then change directory to `/root/sgx/samplecode/hello-rust` and `make`. Then cd to `bin` and `./app`.
+
+## Use docker without hardware support, only with simulation. Windows/Macbook compatible.
+
+Make sure you have docker installed and working.
+
+Start docker as:
+
+```
+$ docker run -ti --rm -v /path/to/sdk:/root/sgx baiduxlab/sgx-rust
+root@913e6a00c8d8:~#
+```
+
+And then build in simulation mode
+```
+$ cd /root/sgx/samplecode/hello-rust
+$ SGX_MODE=SW make
+$ cd bin
+$ ./app
+```
+
+### Use docker with hardware support, and run aesm outside docker (on the host OS)
+
+
+
+Just add another device mapping to the command to have `aesm.socket` works in SGX. This requires step 3 finished on the host OS and `/var/run/aesmd/aesm.socket` exists on the host OS.
+
+```
+$ docker run --rm -ti \
+ --device /dev/isgx \ # forward isgx device
+ -v /path/to/rust-sgx-sdk:/root/sgx \ # add SDK
+ -v /var/run/aesmd:/var/run/aesmd \ # forward domain socket
+ baiduxlab/sgx-rust
+```
+
+Then you can skip launching `aesmd` in the docker container.
+
+# CI setup
+
+The only known solution:[drone.io](http://drone.io) is provided by @elichai. We've set it up successfully.
diff --git a/documents/mitigation-of-intel-sa-00219-in-rust-sgx.md b/documents/mitigation-of-intel-sa-00219-in-rust-sgx.md
new file mode 100644
index 0000000..eed25cb
--- /dev/null
+++ b/documents/mitigation-of-intel-sa-00219-in-rust-sgx.md
@@ -0,0 +1,379 @@
+---
+permalink: /sgx-sdk-docs/mitigation-of-intel-sa-00219
+---
+# Background
+
+Intel issued [Intel SA-00219](https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00219.html) on Nov 12, 2019, with CVE number [CVE-2019-0117](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0117). Intel also published a [guidance](https://software.intel.com/en-us/download/intel-sgx-sdk-developer-guidance-intel-sa-00219) to instruct the developers/researchers. Then Intel released [Intel SGX SDK v2.7.1](https://01.org/intel-softwareguard-extensions/downloads/intel-sgx-linux-2.7.1-release-version-string-2.7.101.3), including new memory allocation primitives and corresponding patches in PSW enclaves.
+
+This article is to help people understand Intel-SA-00219, and how Rust SGX SDK handles it. Please feel free to reach me at dingelish@gmail.com, or dingyu@apache.org.
+
+## The problem statement and my thoughts
+
+The only statement I found is on the [Intel-SA-00219 page](https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00219.html):
+
+> Organize the code/data within enclave memory to avoid putting sensitive materials in DWORD0 and DWORD1 of cache line. The effectiveness of this mitigation is dependent on the ability for the software to avoid the affected memory region. To assist the enclave application providers to modify their code, Intel is releasing SGX SDK update (Windows version 2.5.101.3, Linux version 2.7.101.3) with new memory allocation APIs to avoid the affected memory region. More details about the APIs can be found [here](https://software.intel.com/en-us/download/intel-sgx-sdk-developer-guidance-intel-sa-00219).
+
+Intel does not directly describe the vulnerability here. But it's clear that the 64-byte cache line would contain 8-byte or sensitive data, which can be keys protected by Intel SGX. So the following memory layout can be problematic in SGX:
+
+```
+ --------------------------------------------------------------------------------------
+| attacker accessible data A | private key (inaccessible) | attacker accessible data B |
+ --------------------------------------------------------------------------------------
+```
+
+It's equal to a vulnerable data structure like:
+
+```
+struct foo {
+ uint64_t A;
+ uint64_t secret;
+ uint64_t B;
+}
+```
+
+where `foo.A` and `foo.B` are accessible by design, while `foo.secret` is not.
+
+If an attacker somehow can access either A or B, he probably will have first or last 8-byte of the "inaccessible" secret in cache line. Then something bad may happen.
+
+So, the most straightforward mitigation is to insert additional "guard bytes" before and after the sensitive data:
+
+```
+ ----------------------------------------------------------------------------------------------
+| attacker data A | 8-byte guard | private key (inaccessible) | 8-byte guard | attacker data B |
+ ----------------------------------------------------------------------------------------------
+```
+
+It results in a modified structure like
+
+```
+struct foo {
+ uint64_t A;
+ (private) uint64_t _guard0;
+ uint64_t secret;
+ (private) uint64_t _guard1;
+ uint64_t B;
+}
+```
+
+Further investigation from Intel's code reveals that `_guard1` is not required. So it can be:
+
+```
+ -------------------------------------------------------------------------------
+ | attacker data A | 8-byte guard | private key (inaccessible) | attacker data B |
+ -------------------------------------------------------------------------------
+```
+
+## Intel's new allocator primitive
+
+Intel's guidance provides:
+
+(1) A C++ template `custom_alignment_aligned`
+(2) A C function `sgx_get_aligned_ptr` and one of its parameter's type `struct align_req_t`
+(3) A dynamic memory allocator function `sgx_aligned_malloc`
+
+After spending hours on Intel's code, I realized that these primitives are helping developers allocate a larger object which:
+
+a) contains all fields of the original object.
+b) adds "guard bytes" before and after each "specified secret field".
+c) align each "specified secret field" on demand
+
+## Intel's patches on PSW enclaves
+
+The most easy to understand example is from `psw/ae/pse_op/session_mgr.cpp`:
+
+```diff
+@@ -417,7 +461,12 @@ pse_op_error_t pse_exchange_report(uint64_t tick,
+ {
+ pse_op_error_t status = OP_SUCCESS;
+ sgx_dh_session_t sgx_dh_session;
+- sgx_key_128bit_t aek;
++ //
++ // securely align aek
++ //
++ //sgx_key_128bit_t aek;
++ sgx::custom_alignment_aligned<sgx_key_128bit_t, sizeof(sgx_key_128bit_t), 0, sizeof(sgx_key_128bit_t)> oaek;
++ sgx_key_128bit_t& aek = oaek.v;
+ sgx_dh_session_enclave_identity_t initiator_identity;
+ cse_sec_prop_t * pcse_sec = NULL;
+ secu_info_t* psec_info = NULL;
+```
+
+The template generates a larger struct `oaek`. Size of `sgx_key_128bit_t` is 16 bytes, and `sizeof(oaek)` equals to 32. And the offset of `oaek.v` is 8.
+
+And in the same file, another fix is:
+
+```diff
+--- a/psw/ae/pse/pse_op/session_mgr.cpp
++++ b/psw/ae/pse/pse_op/session_mgr.cpp
+@@ -29,21 +29,65 @@
+ *
+ */
+
+-
++#include <sgx_secure_align.h>
+ #include "utility.h"
+ #include "session_mgr.h"
+ #include "pse_op_t.h"
+ #include "sgx_dh.h"
+
+ // ISV enclave <-> pse-op sessions
+-static pse_session_t g_session[SESSION_CONNECTION];
++//
++// securely align all ISV enclave - pse sessions' secrets
++//
++static sgx::custom_alignment_aligned<pse_session_t, 16, __builtin_offsetof(pse_session_t, active.AEK), 16> og_session[SESSION_CONNECTION];
++//
++// following allows existing references to g_session[index]
++// to not have to change
++//
++class CSessions
++{
++public:
++ pse_session_t& operator[](int index) {
++ return og_session[index].v;
++ }
++};
++static CSessions g_session;
+ static uint32_t g_session_count = 0;
+```
+
+It seems that the original global `g_session` array is vulnerabile to INTEL-SA-00219. So Intel created a new structure `CSessions` and reloaded the `[]` operator, and used `custom_alignment_aligned` template to create the array of guarded `CSessions`.
+
+We can see some more complex samples in the same file, such as:
+
+```diff
+ // ephemeral session global variables
+ static uint8_t g_nonce_r_pse[EPH_SESSION_NONCE_SIZE] = {0}; // nonce R(PSE) for ephemeral session establishment
+ static uint8_t g_nonce_r_cse[EPH_SESSION_NONCE_SIZE] = {0}; // nonce R(CSE) for ephemeral session establishment
+-static pairing_data_t g_pairing_data; // unsealed pairing data
+-eph_session_t g_eph_session; // ephemeral session information
++
++//
++// securely align pairing data
++// Id_pse and Id_cse aren't secrets
++// I don't think pairingNonce is a secret and even if it is, we can't align
++// all of [mk, sk, pairingID, pairingNonce]
++//
++//static pairing_data_t g_pairing_data; // unsealed pairing data
++static sgx::custom_alignment<pairing_data_t,
++ //__builtin_offsetof(pairing_data_t, secret_data.Id_pse), sizeof(((pairing_data_t*)0)->secret_data.Id_pse),
++ //__builtin_offsetof(pairing_data_t, secret_data.Id_cse), sizeof(((pairing_data_t*)0)->secret_data.Id_cse),
++ __builtin_offsetof(pairing_data_t, secret_data.mk), sizeof(((pairing_data_t*)0)->secret_data.mk),
++ __builtin_offsetof(pairing_data_t, secret_data.sk), sizeof(((pairing_data_t*)0)->secret_data.sk),
++ __builtin_offsetof(pairing_data_t, secret_data.pairingID), sizeof(((pairing_data_t*)0)->secret_data.pairingID)
++ //__builtin_offsetof(pairing_data_t, secret_data.pairingNonce), sizeof(((pairing_data_t*)0)->secret_data.pairingNonce)
++ > opairing_data;
++pairing_data_t& g_pairing_data = opairing_data.v;
++//
++// securely align pse - cse/psda ephemeral session secrets
++//
++//eph_session_t g_eph_session; // ephemeral session information
++sgx::custom_alignment<eph_session_t,
++ __builtin_offsetof(eph_session_t, TSK), sizeof(((eph_session_t*)0)->TSK),
++ __builtin_offsetof(eph_session_t, TMK), sizeof(((eph_session_t*)0)->TMK)
++> oeph_session;
++//
++// this reference trick requires change to declaration
++// in other files, but still cleaner than changing
++// all references
++//
++eph_session_t& g_eph_session = oeph_session.v;
+
+ /**
+ * @brief Check the status of the ephemeral session
+```
+
+To understand it, let me expand `struct pairing_data_t` here:
+
+```
+/* Pairing blob unsealed and usable inside of enclave*/
+typedef struct _pairing_data_t
+{
+ se_plaintext_pairing_data_t plaintext; // does not involved
+ struct se_secret_pairing_data_t {
+ SHA256_HASH Id_pse;
+ SHA256_HASH Id_cse;
+ SIGMA_MAC_KEY mk;
+ SIGMA_SECRET_KEY sk;
+ SIGMA_SECRET_KEY pairingID; // old_sk used for repairing check
+ Nonce128_t pairingNonce;
+ EcDsaPrivKey VerifierPrivateKey;
+ } secret_data;
+} pairing_data_t;
+```
+
+The patch seems to protect `mk`, `sk`, and `pairingID`, and all the other fields are commented out. What's more, this patch uses a **undocumented** template `sgx::custom_alignment` defined as:
+
+```cpp
+ template <class T, std::size_t... OLs>
+ using custom_alignment = custom_alignment_aligned<T, alignof(T), OLs...>;
+```
+
+## Experiments on the undocument template
+
+To test how the undocumented template work, I write the following codes:
+
+```cpp
+ struct foo {
+ uint64_t secret1[5]; // offset = 0
+ };
+
+ typedef sgx::custom_alignment<foo, __builtin_offsetof(foo, secret1), sizeof(((foo*)0)->secret1)> AFOO;
+
+ printf("=== Size of foo = %u ===\n", sizeof(foo)); // 40
+ printf("=== Size of bar = %u ===\n", sizeof(AFOO)); // 64
+ printf("=== offset of AROO.v = %u ===\n", __builtin_offsetof(AFOO, v)); // 8
+ printf("=== offset of secret1 = %u ===\n", __builtin_offsetof(AFOO, v.secret1)); // 8
+```
+
+So we can see that the structure of AROO is:
+
+```cpp
+struct AROO {
+ uint64_t _padding_head[1] // offset = 0, len = 8
+ struct {
+ uint64_t secret1[5]; // offset = 8, len = 40
+ } v;
+ uint64_t _padding_tail[2]; // offset = 40, len = 16
+```
+
+It seems the undocumented C++ template aligns `AROO` to the next level, and add 8-byte headings into it. If we add the second secret in `foo` like:
+
+```cpp
+ struct foo {
+ uint64_t secret1[5]; // offset = 0
+ uint64_t secret2[1]; // offset = 40
+ };
+
+ typedef sgx::custom_alignment<foo,
+ __builtin_offsetof(foo, secret1), sizeof(((foo*)0)->secret1),
+ __builtin_offsetof(foo, secret2), sizeof(((foo*)0)->secret2)
+ > AFOO;
+
+ printf("=== Size of foo = %u ===\n", sizeof(foo)); // 48
+ printf("=== Size of bar = %u ===\n", sizeof(AFOO)); // 64
+ printf("=== offset of AROO.v = %u ===\n", __builtin_offsetof(AFOO, v)); // 8
+ printf("=== offset of AROO.v.secret1 = %u ===\n", __builtin_offsetof(AFOO, v.secret1)); // 8
+ printf("=== offset of AROO.v.secret2 = %u ===\n", __builtin_offsetof(AFOO, v.secret2)); // 48
+```
+
+we can see that the structure of AROO is:
+
+```cpp
+struct AROO {
+ uint64_t _padding_head[1] // offset = 0, len = 8
+ struct {
+ uint64_t secret1[5]; // offset = 8, len = 40
+ uint64_t secret2[1]; // offset = 48, len = 8
+ } v;
+ uint64_t _padding_tail[1]; // offset = 56, len = 8
+```
+
+If we increase `secret2` to 16-bytes, it works well as usual. And the `_padding_tail` will have **zero length**. So does it means that *only extra heading is required for mitigation*? But it'll not compile if we make `secret2` 24-bytes, like:
+
+```c++
+ struct foo {
+ uint64_t secret1[5]; // offset = 0
+ uint64_t secret2[3]; // offset = 40
+ };
+
+ typedef sgx::custom_alignment<foo,
+ __builtin_offsetof(foo, secret1), sizeof(((foo*)0)->secret1),
+ __builtin_offsetof(foo, secret2), sizeof(((foo*)0)->secret2)
+ > AFOO;
+```
+
+GCC would terminate on:
+
+```
+make[1]: Entering directory '/root/linux-sgx/SampleCode/Cxx11SGXDemo'
+In file included from Enclave/TrustedLibrary/Libcxx.cpp:47:0:
+/opt/sgxsdk/include/sgx_secure_align.h: In instantiation of 'struct sgx::__custom_alignment_internal::custom_alignment<ecall_lambdas_demo()::foo, 8ul, -1>':
+Enclave/TrustedLibrary/Libcxx.cpp:125:53: required from here
+/opt/sgxsdk/include/sgx_secure_align.h:123:13: error: static assertion failed: No viable offset
+ static_assert(LZ > 0, "No viable offset");
+ ^
+/opt/sgxsdk/include/sgx_secure_align.h:125:48: error: size of array is negative
+ char __no_secret_allowed_in_here[LZ];
+ ^
+Makefile:255: recipe for target 'Enclave/TrustedLibrary/Libcxx.o' failed
+```
+
+Nothing changes if we switch to the original template `sgx::custom_alignment_aligned`. So I guess the template does not support structures:
+
+- contains secrets consecutively, and
+- the consecutive secrets' size is larger than a certain number (not sure yet)
+
+If we break down `secret1` and `secret2` by inserting something in the middle, the template works:
+
+```c++
+struct foo {
+ uint64_t secret1[5]; // offset = 0
+ char dumb; // offset = 40
+ uint64_t secret2[3]; // offset = 48
+};
+
+typedef sgx::custom_alignment<foo,
+__builtin_offsetof(foo, secret1), sizeof(((foo*)0)->secret1),
+__builtin_offsetof(foo, secret2), sizeof(((foo*)0)->secret2)
+ > AFOO;
+
+printf("=== Size of foo = %u ===\n", sizeof(foo)); // 72
+printf("=== Size of bar = %u ===\n", sizeof(AFOO)); // 128
+printf("=== offset of AROO.v = %u ===\n", __builtin_offsetof(AFOO, v)); // 24
+printf("=== offset of AROO.v.secret1 = %u ===\n", __builtin_offsetof(AFOO, v.secret1)); // 24
+printf("=== offset of AROO.v.secret2 = %u ===\n", __builtin_offsetof(AFOO, v.secret2)); // 72
+```
+
+## Changes/Actions required
+
+From Intel's usage, we can learn that:
+
+**Don't construct a sensitive data structure directly. Always allocate an aligned structure and fill it up later **
+
+It means:
+
+* if you allocate something sensitive (e.g. keys in `sgx_key_128bit_t`) on stack/heap, you probably need to allocate another guarded structure first, and get a mutable reference to its inner data.
+* if you want to make `sgx_key_128bit_t` as the type of return value, you can choose between (1) return a guarded structure, or (2) takes an additional argument of caller-allocated, mutuable reference of `sgx_key_128bit_t` and fill it.
+
+## Rust SGX provided primitive
+
+* We provided `AlignBox` as a replacement of `Box`
+
+ * `Box` is somewhat tricky because it always "initialize on stack first and copy to heap later". [copyless](https://github.com/kvark/copyless) provides a novel primitive to solve [it but it does not always effective](https://github.com/dingelish/realbox). To this end, we created `AlignBox` which guarantees "on-heap initialization" without copying any bits. Usage:
+
+ ```rust
+ let heap_align_obj = AlignBox::<struct_align_t>::heap_init_with_req(|mut t| {
+ t.key1 = [0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff];
+ t.pad1 = [0x00; 16];
+ t.key2 = [0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff];
+ t.pad2 = [0x00; 16];
+ t.key3 = [0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff];
+ t.pad3 = [0x00; 16];
+ t.key4 = [0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff];
+ }, 16, &str_slice);
+ assert!(heap_align_obj.is_some());
+ ```
+
+* We provided aligned key type for each built-in key type. The layout are calculated by Intel's template.
+
+ * `sgx_align_key_128bit_t`
+ * `sgx_align_mac_128bit_t`
+ * `sgx_align_key_256bit_t`
+ * `sgx_align_mac_256bit_t`
+ * `sgx_align_ec256_dh_shared_t`
+ * `sgx_align_ec256_private_t`
+
+We modified `sgx_tcrypto`, `sgx_tse`, and `sgx_tdh` and use the above primitives for enhancement, following the above required changes. One sample is from `sgx_tcrypto`:
+
+```rust
++ let mut align_mac = sgx_align_mac_128bit_t::default();
++ let ret = unsafe {
++ sgx_rijndael128_cmac_msg(key as * const sgx_cmac_128bit_key_t,
++ src.as_ptr() as * const u8,
++ size as u32,
++ &mut align_mac.mac as * mut sgx_cmac_128bit_tag_t)
++ };
+```
+
+We allocate an aligned structure first, and then fill it up using Intel's crypto primitive later.
diff --git a/documents/performance-optimization-tips.md b/documents/performance-optimization-tips.md
new file mode 100644
index 0000000..1f827cf
--- /dev/null
+++ b/documents/performance-optimization-tips.md
@@ -0,0 +1,52 @@
+---
+permalink: /sgx-sdk-docs/performance-optimization-tips
+---
+# Performance Optimization Tips
+
+## Enable link time optimization
+
+This could boost CPU-intensive enclaves about 2~5% (on my 9900K)
+
+In `Cargo.toml`:
+```toml
+[profile.release]
+lto = true
+```
+
+## Let rustc emit asm
+
+This could boost enclaves on some platforms.
+
+Set an environment variable as
+
+```bash
+export RUSTFLAGS="--emit asm"
+```
+
+Or create a `.cargo/config` which covers your project as:
+
+```toml
+[build]
+rustflags = ["--emit","asm"]
+```
+
+## Configure target_cpu for llvm
+
+This could boost enclaves on some platforms (not effective on my 9900K).
+
+Set an environment variable as
+
+```bash
+export RUSTFLAGS="-C target-cpu=native"
+```
+
+Or create a `.cargo/config` which covers your project as:
+
+```toml
+[build]
+rustflags = ["-C", "target-cpu=native"]
+```
+
+## Enable lto on the final linking step
+
+Add `-flto` to the final linking step using `CXX`.
diff --git a/documents/public-cloud-for-rust-sgx-development.md b/documents/public-cloud-for-rust-sgx-development.md
new file mode 100644
index 0000000..6a1a633
--- /dev/null
+++ b/documents/public-cloud-for-rust-sgx-development.md
@@ -0,0 +1,9 @@
+---
+permalink: /sgx-sdk-docs/public-cloud-for-rust-sgx-dev
+---
+# List of public available cloud service provide for Rust SGX development
+
+- [Azure Confidential Computing DCsV2 Node](https://azure.microsoft.com/en-us/updates/intel-sgx-based-confidential-computing-vms-now-available-on-azure-dedicated-hosts/)
+- [SecretVPS](http://www.secretvps.io/rust-sgx)
+
+More and more SGX2 platforms are coming up in 2021!
diff --git a/documents/setup-gdb-7.11-on-ubuntu-18.04-for-vscode---sgx-gdb-remote-debugging.md b/documents/setup-gdb-7.11-on-ubuntu-18.04-for-vscode---sgx-gdb-remote-debugging.md
new file mode 100644
index 0000000..f5bce13
--- /dev/null
+++ b/documents/setup-gdb-7.11-on-ubuntu-18.04-for-vscode---sgx-gdb-remote-debugging.md
@@ -0,0 +1,70 @@
+---
+permalink: /sgx-sdk-docs/setup-gdb-ubuntu18
+---
+This instruction is provided by @akoskinas. Thanks!
+This is an updated version of the process, which contains bug fixes and additional comments.
+
+------------------------------
+Before start: In our setup, the debugging works only when building in simulation mode, i.e
+```
+cd rust-sgx-sdk/samplecode/hello-rust-vscode-debug/
+SGX_MODE=SW SGX_DEBUG=1 make
+cd bin
+sgx-gdb ./app
+```
+A short description of the process needed in order to use GDB to remotely debug a Rust SGX enclave with sgx gdb in Ubuntu 18.04:
+
+1. Make sure that the package libsgx-enclave-common-dbgsym_${version}-${revision}_amd64.ddeb is installed, as described here: https://github.com/intel/linux-sgx#build-the-intelr-sgx-psw-installer . The package can also be found here: https://download.01.org/intel-sgx/linux-2.5/ubuntu18.04-server/
+
+2. Make sure to set up the needed environment variables before compiling your code. To do so, run:
+```
+ $ source ${sgx-sdk-install-path}/environment
+```
+3. As documented [here](debugging-a-local-rust-sgx-enclave-in-docker-with-sgx-gdb.md) , an older version of GDB debugger has to be utilized for debugging. The steps to use gdb-7.11.1 are
+
+- get the source code of version 7.11.1 :
+```
+wget "http://ftp.gnu.org/gnu/gdb/gdb-7.11.1.tar.gz"
+```
+
+- extract
+```
+tar -xvzf gdb-7.11.1.tar.gz
+```
+- install the python development headers, needed to configure the GDB python interpreter:
+```
+sudo apt-get install python3-dev
+```
+
+- configure the build: because python scripts will be given as input to the GDB, a python interpreter has to be configured at this step. To do so the option "with-python" shall be used, followed by the path to the desired python version
+```
+cd gdb-7.11.1
+./configure --with-python=/usr/bin/python3
+```
+
+- build: in order for building to complete the following changes are required to solve a type conflict - building takes ~3mins :
+
+In file: gdb/amd64-linux-nat.c:248 --> delete word "const"
+in file: gdb/gdbserver/linux-x86-low.c:239:1 --> delete word "const"
+```
+make
+```
+
+- Two options are available to complete installation:
+
+If GDB 7.11.1 is desired to be located in /usr/bin/gdb then execute:
+
+```
+sudo make install
+```
+If GDB 7.11.1 is desired to be located in a different location, two symbolic links are needed. In our case, we chose to place gdb-7.11.1 folder under /opt directory. In that case the respective commands will look like the following:
+```
+ sudo ln -sf /opt/gdb-7.11.1/gdb/gdb /usr/bin/gdb
+ cd /usr/local/share
+ # if gdb dir doesn't exist, create it : mkdir -p gdb
+ cd gdb
+ # if python dir doesn't exist, create it: mkdir -p python
+ cd python
+ sudo ln -s /opt/gdb-7.11.1/gdb/data-directory/python/gdb/ /usr/local/share/gdb/python/
+```
+4. Up until this point, local debugging should be successful. The final step is to use VS Code, by following the steps described [here](use-vscode---rls---rust-analysis---sgx-gdb-for-graphic-developing-(not-in-docker).
diff --git a/documents/sgx_in_mesalock_linux.md b/documents/sgx_in_mesalock_linux.md
index 5a5c835..81da331 100644
--- a/documents/sgx_in_mesalock_linux.md
+++ b/documents/sgx_in_mesalock_linux.md
@@ -1,3 +1,6 @@
+---
+permalink: /sgx-sdk-docs/sgx_in_mesalock_linux
+---
# Run Rust SGX Applications in Mesalock Linux
MesaLock Linux is a general purpose Linux distribution which aims to provide a safe and secure user space environment. To eliminate high-severe vulnerabilities caused by memory corruption, the whole user space applications are rewritten in memory-safe programming languages like Rust and Go. This extremely reduces attack surfaces of an operating system exposed in the wild, leaving the remaining attack surfaces auditable and restricted. Therefore, MesaLock Linux can substantially improve the security of the Linux ecosystem. Additionally, thanks to the Linux kernel, MesaLock Linux supports a broad hardware environment, making it deployable in many places. Two main usage scenarios of MesaLock Linux are for containers and security-sensitive embedded devices. With the growth of the ecosystem, MesaLock Linux would also be adopted in the server environment in the future.
@@ -8,7 +11,7 @@
A typical Rust SGX application has at least to components: one enclave, and one untrusted component. The enclave is self-contained and doesn't need dynamic loading. The untrusted component depends on `liburts` (untrusted runtime service library), which depends on the Application Enclave Services Manager library.
-We show the dependency tree as follows. In this tree, we hide all the dynamic libraries which already exist in Mesalock Linux.
+We show the dependency tree as follows. In this tree, we hide all the dynamic libraries which already exist in Mesalock Linux.
```bash
SGX Application
@@ -28,7 +31,7 @@
1. Unsafe components must not taint safe components, especially for public APIs and data structures.
2. Unsafe components should be as small as possible and decoupled from safe components.
-3. Unsafe components should be explicitly marked during deployment and ready to upgrade.
+3. Unsafe components should be explicitly marked during deployment and ready to upgrade.
Hence, we believe that running Rust SGX applications in Mesalock Linux could provide better security guarantees if they follow the memory safety principles.
@@ -44,7 +47,7 @@
There are two options for running the aesm service: (1) start `aesmd` inside the Mesalock Linux container, or (2) start `aesmd` on the host OS and provide service to the SGX application inside the container. The first method provides better isolation for `aesmd`, but it would start a set of infrastructure enclaves for each docker container, wasting the limited EPC memory. The second one only launches one set of infrastructure enclaves for all SGX containers and we believe it is more efficient.
-In our current solution, we put the AESM service process `aesmd` outside the Mesalock Linux docker container and only expose the domain socket `/var/run/aesm/aesm.socket` to the container. In this way, we isolate the AESM service along with the six foundation enclaves (Launch Enclave/Quoting Enclave/Provisioning Enclave/Provisioning Certification Enclave/Platform Service Enclave for long term pairing/Platform Service Enclave for session management) from Mesalock Linux docker container.
+In our current solution, we put the AESM service process `aesmd` outside the Mesalock Linux docker container and only expose the domain socket `/var/run/aesm/aesm.socket` to the container. In this way, we isolate the AESM service along with the six foundation enclaves (Launch Enclave/Quoting Enclave/Provisioning Enclave/Provisioning Certification Enclave/Platform Service Enclave for long term pairing/Platform Service Enclave for session management) from Mesalock Linux docker container.
@@ -113,5 +116,3 @@
| libsgx_urts.so | 166376 | 05a15c27432bded48b49b98f94649b4e90abaedbb4ce8f3c44baa1ff5ce2493d | [sgx_linux_ubuntu16.04.1_x64_psw_2.0.100.40950.bin](https://download.01.org/intel-sgx/linux-2.0/sgx_linux_ubuntu16.04.1_x64_psw_2.0.100.40950.bin) |
| libstdc++.so.6 | 1594800 | 5e68ec758e36fb2db90f6be673cf4112d144a2f29ba400cd5c6e0c8e56ad9408 | [libstdc++6_7.2.0-1ubuntu1~16.04_amd64.deb](https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+files/libstdc++6_7.2.0-1ubuntu1~16.04_amd64.deb) |
| libz.so.1 | 104864 | a04cab74df9c7f96f82b34286bda5d4ee810feaac92dd2e8bcfe931d9c8baef4 | [zlib1g_1.2.11.dfsg-0ubuntu1_amd64.deb](http://us.archive.ubuntu.com/ubuntu/pool/main/z/zlib/zlib1g_1.2.11.dfsg-0ubuntu1_amd64.deb) |
-
-
diff --git a/documents/sgxtime.md b/documents/sgxtime.md
index aad3aea..c6bcea1 100644
--- a/documents/sgxtime.md
+++ b/documents/sgxtime.md
@@ -1,3 +1,6 @@
+---
+permalink: /sgx-sdk-docs/sgxtime
+---
# Acquiring Trusted timestamp from Intel ME in SGX enclave
Intel provides Trusted Platform Service via Intel Management Engine. Part of the Intel SGX features depend on the trusted platform service, including the trusted timestamp service. We implemented `sgx_tservice::SgxTime` for this feature. To use it in Linux, the prerequisites are:
@@ -117,4 +120,3 @@
Next, uninstall the current PSW (if installed) and reinstall it.
Now, `sgxtime` should work.
-
diff --git a/documents/the-world-of-forked-crates.md b/documents/the-world-of-forked-crates.md
new file mode 100644
index 0000000..a30b306
--- /dev/null
+++ b/documents/the-world-of-forked-crates.md
@@ -0,0 +1,40 @@
+---
+permalink: /sgx-sdk-docs/the-world-of-forked-crates
+---
+# The World of Forked crates
+
+We created a "super repo" at [dingelish/sgx-world](https://github.com/dingelish/sgx-world), which includes all the forked repos as git submodules.
+
+We created a "dumb crate" [dumb-all](https://github.com/dingelish/sgx-world/tree/master/dumb-all) to help us maintain the world of forked crates. Each dependency of dumb-all is a valid crate in Rust-SGX ecosystem. The enabled feature set is likely to be the universe of supported feature set. Exceptions exist in a couple of crates, where there are conflict features.
+
+Generally speaking, to use a forked crate, simply use the following statement:
+
+```toml
+wasmi = { git = "https://github.com/mesalock-linux/wasmi-sgx" }
+```
+
+No need to config the branch in most cases. Only a few of the forked crates use specific branches.
+
+## Tagging
+
+As of today, Oct 23 2019, cargo still has problem on git dependencies. The issue is [cargo #7497](https://github.com/rust-lang/cargo/issues/7497). It leads to a dependency hell if the Cargo.toml looks like:
+
+```toml
+foo = { git = "https://github.com/..." }
+foo = { git = "https://github.com/...", rev = "deadbeaf" }
+foo = { git = "https://github.com/...", tag = "deadbeaf" }
+```
+
+These three foos are all valid, but different from each other in compilation. Assume that crate `bar` depends on foo with the first one, and `haa` depends on foo with the second one, then the dependency tree is going to mess up. So this is the reason that almost all forked crates are tagged with `sgx_1.0.9` but the cross references are not using that tag.
+
+One potential fix is a tool to fix Cargo.lock and remove the redundant crates. I've done this successfully by manually dedup Cargo.lock according to the hash value instead of URI. Maybe we can automate this later"
+
+## The World Map
+
+Generated by `cargo deps` on dumb-all.
+
+
+
+## Security audit
+
+We use [`cargo-audit`](https://github.com/RustSec/cargo-audit) to automatically check for known vulnerabilities. And the pipeline has been added to CI which runs nightly. We'll update immediately once we find a security issue by `cargo-audit`. Thanks to @tarcieri for this fantastic tool!
diff --git "a/documents/use-vscode---rls---rust-analysis---sgx-gdb-for-graphic-developing-\050not-in-docker\051.md" "b/documents/use-vscode---rls---rust-analysis---sgx-gdb-for-graphic-developing-\050not-in-docker\051.md"
new file mode 100644
index 0000000..f82f9cd
--- /dev/null
+++ "b/documents/use-vscode---rls---rust-analysis---sgx-gdb-for-graphic-developing-\050not-in-docker\051.md"
@@ -0,0 +1,81 @@
+---
+permalink: /sgx-sdk-docs/use-vscode-rls
+---
+This is my personal setup and contains some IP/path/usernames. Please tweak them in your environment.
+
+
+
+# Solution overview
+1. Use Visual Code Insider and the [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) plugin to establish a vscode-ssh session.
+2. Tweak a project with a new `Cargo.toml` workspace and all `Makefile`s. This enables `rls`.
+3. Tweak the build options for compiling in debug mode.
+4. Use [Native Debug](https://marketplace.visualstudio.com/items?itemName=webfreak.debug) plugin for graphic debugging.
+
+# Prerequisites
+
+* Visual Code Insider installed on your machine. OS is flexible.
+* Remote Linux supports Intel SGX, with SSH service started.
+* `rustup`, Intel SGX driver/PSW/SDKs are correctly installed. `hello-rust` code sample works.
+* Remote Linux **could** be the same machine. Just ignore the `vscode-ssh` plugin mentioned in this wiki page and you'll be fine.
+
+My personal setup is:
+* Macbook. MacOS 10.14.4 + VSCode Insider.
+* Remote desktop PC running Ubuntu Linux 18.04. Intel SGX v2.5.
+
+# Known bugs
+* `sgx-gdb` throws Python exception on `gdb` > 7.12 on some platforms, such as mine. But native sgx-gdb may not throw that error. Don't have a solution for VSCode yet. YMMV.
+
+# Steps
+
+## Setup the vscode-ssh session.
+0. Setup a convenient way for ssh login. I always append my `~/.ssh/id_rsa.pub` to the remote `~/.ssh/authorized_keys`.
+1. Install the [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) plugin.
+2. Establish a vscode-ssh session to the remote Linux.
+
+## Create an rls-friendly rust-sgx project.
+
+[hello-rust-vscode-debug](https://github.com/apache/incubator-teaclave-sgx-sdk/tree/master/samplecode/hello-rust-vscode-debug) is an example. Differences between this and `hello-rust` are:
+1. An extra `Cargo.toml` at the root, which contains two Rust crates: `app` and `enclave`. This change would result in changing the path of `target` folders.
+2. Tweak `Makefile` and `enclave/Makefile` and correct the path of `target` folders.
+3. Tweak `Makefile` and `enclave/Makefile` to enable debug compilation. Changes include: (1) remove `--release` in `cargo build`, (2) add `-ggdb` to `SGX_COMMON_FLAGS`.
+
+After these steps, the `hello-rust-vscode-debug` should be an rls-friendly project. And open the remote folder of it in the VSCode main screen "Start - open folder". Then autocompletion should work!
+
+## Setup Native Debug with sgx-gdb
+
+Now we have a vscode-ssh session to the remote Linux and an opened folder of `hello-rust-vscode-debug`. The next step is to configure a correct `launch.json` for Native Debug plugin. Now open the debug panel of VS code and click on the gear icon to open `launch.json` in the editor.
+```
+{
+ // Use IntelliSense to learn about possible attributes.
+ // Hover to view descriptions of existing attributes.
+ // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+ "version": "0.2.0",
+ "configurations": [
+ {
+ "name": "Debug",
+ "type": "gdb",
+ "request": "launch",
+ "target": "app",
+ "cwd": "${workspaceRoot}/bin",
+ "valuesFormatting": "parseText",
+ "gdbpath": "sgx-gdb",
+ "ssh": {
+ "forwardX11": false,
+ "host": "172.19.32.44", // your IP
+ "cwd": "${workspaceRoot}/bin",
+ // SSH private key on remote machine. Add the pub key to ~/.ssh/authorized_keys
+ // This ssh configuration is established from host to host, because the current
+ // vscode session is "within a ssh session established by vscode-ssh".
+ // I think this might be a bug but can hardly be resolved.
+ "keyfile": "/home/ding/.ssh/id_rsa", // private key
+ "user": "ding",
+ "bootstrap": "source /opt/sgxsdk/environment",
+ "port": 22
+ }
+ }
+ ]
+}
+```
+`name`,`type`,`request`,`valuesFormatting` are default values. `cwd` is the working directory we launch the app, so it should be the `bin` folder. `target` is the debugee executable so it should be the `app`. `host` is the IP address of your Linux machine. Then comes the tricky part: ssh. It means that we use an extra ssh session for debugger, within the current vscode-ssh session. This means that we are here creating an additional ssh session from remote machine to itself. Only in this way could we setup the environment using the Intel's script before launching `sgx-gdb`. So we need to add the public key `~/.ssh/id_rsa` to `~/.ssh/authorized_keys` and demonstrate the corresponding private key as `~/.ssh/id_rsa`.
+
+Having this `launch.json` configured correctly, we could simply set up a breakpoint on the first line of `say_something` and start debugging. Enjoy!
diff --git a/sgx_no_tstd/build.rs b/sgx_no_tstd/build.rs
index 2fea965..c6e8bd8 100644
--- a/sgx_no_tstd/build.rs
+++ b/sgx_no_tstd/build.rs
@@ -35,6 +35,7 @@
"autom4te.cache",
"Makefile.in",
"config.h.in",
+ "config.h.in~",
"configure",
"aclocal.m4",
"INSTALL"];
@@ -80,4 +81,4 @@
.arg(format!("INCDIR={}", native.src_dir.display()))
.arg("-j5"));
Ok(())
-}
\ No newline at end of file
+}
diff --git a/sgx_unwind/build.rs b/sgx_unwind/build.rs
index 0f64ebd..1f3f228 100644
--- a/sgx_unwind/build.rs
+++ b/sgx_unwind/build.rs
@@ -35,6 +35,7 @@
"autom4te.cache",
"Makefile.in",
"config.h.in",
+ "config.h.in~",
"configure",
"aclocal.m4",
"INSTALL"];
