Presentation Slides: Second Order Sensitivities: AAD Construction ...

Second Order Sensitivities: AAD Construction and use for CPU and GPU

October 2017 Jacques du Toit and Chris Kenyon

Experts in numerical software and High Performance Computing

Acknowledgements

This is joint work with Chris Kenyon

High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com

2

First and Second Order Adjoint Models


3

First Order Adjoint Model For a function 𝐹 ∶ ℝ𝑚 → ℝ 𝑛 the first order adjoint model is 𝑇

𝜕𝐹 ) 𝑦(1) 𝜕𝑥 ↑ ↑ ℝ𝑛×1 ℝ𝑛×𝑚

𝑥(1) = (

= 𝐽 𝑇 𝑦(1) Order of complexity for full Jacobian is order of output space, not input space


4

Second Order Adjoint Model For a function 𝐹 ∶ ℝ𝑚 → ℝ 𝑛 the second order adjoint model is 𝑇

(2) 𝑥(1)

=

𝑥(2)

↑ ℝ1×𝑚

𝜕2 𝐹 ( ) 𝑦(1) 𝜕𝑥 ↑ ↑ 𝑛×1 ℝ ℝ𝑛×𝑚×𝑚

= 𝑥(2) 𝐻 𝑇 𝑦(1) Order of complexity for full Hessian is order of input space times order of output space


5

Differentiating Non-Differentiable Functions


6

American Put Option

𝑉(𝑡, 𝑥) =

sup 0≤𝜏≤𝑇−𝑡

𝔼𝑡,𝑥 (𝐾 − 𝑆𝑡+𝜏 )

= 𝔼𝑡,𝑥 (𝐾 − 𝑆𝑡+𝜏𝒟(𝑡,𝑥) )

+

+

where 𝜏𝒟 (𝑡, 𝑥) is first entry time into stopping region 𝒟


7

American Put Option

𝑉(𝑡, 𝑥) =




+

+

where 𝜏𝒟 (𝑡, 𝑥) is first entry time into stopping region 𝒟 ▶

(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒞


7

American Put Option

𝑉(𝑡, 𝑥) =




+

+


(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒞

▶

(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒟 𝑜


7

American Put Option

𝑉(𝑡, 𝑥) =




+

+


(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒞

▶

(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒟 𝑜

▶

(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,1 globally (smooth fit)


7

American Put Option

𝑉(𝑡, 𝑥) =




+

+


(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒞

▶

(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒟 𝑜

▶


▶

𝑥 ↦ 𝑉𝑥𝑥 (𝑡, 𝑥) is discontinuous over exercise boundary


7

American Put Option

𝑉(𝑡, 𝑥) =




+

+


(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒞

▶

(𝑡, 𝑥) ↦ 𝑉(𝑡, 𝑥) is 𝐶1,2 in 𝒟 𝑜

▶


▶

𝑥 ↦ 𝑉𝑥𝑥 (𝑡, 𝑥) is discontinuous over exercise boundary

▶

But limits from left and right exist


7

Bermudan Options ▶

These results generalise (although hard to prove in generality)

▶

Value functions are “smooth” in 𝒞 and 𝒟 𝑜

▶

Value functions are not second order smooth over exercise boundaries


8

Bermudan Options ▶

These results generalise (although hard to prove in generality)

▶

Value functions are “smooth” in 𝒞 and 𝒟 𝑜

▶

▶

▶

▶

Value functions are not second order smooth over exercise boundaries

As long as we don’t differentiate 𝑉 at an exercise boundary, everything is OK Second order finite differences will almost surely cross the boundary and hence fail Very hard to use finite differences for second order


8

Some Real Analysis


9

Differentiating Under the Integral

𝐹 = ∫ 𝑓 𝑑𝑧

does not mean

𝐹′ = ∫ 𝑓 ′ 𝑑𝑧

However our Monte Carlo code invariably is 𝐹 = ∫ 𝑓 𝑑𝑧 ≈

1 ∑ 𝑓𝑛 (𝑧𝑖 ) 𝑁

where 𝑓𝑛 → 𝑓

We’re forced to differentiate under the integral whether we want to or not


10

Pointwise Convergence ▶ ▶

𝑓𝑛 → 𝑓 does not mean 𝑓𝑛′ → 𝑓 ′

If 𝑓 ′ exists but 𝑓𝑛′ doesn’t, why would 𝑓𝑛′ → 𝑓 ′ ?

Our code is typically not differentiable ... if( xt > barrier (t,xt) ) { sum += payoff (t, xt); continue ; } ...

1 2 3 4 5 6

▶

This is a problem for first and second order sensitivities!


11

Smoothing


12

Vibrato Monte Carlo ▶

▶

▶

▶

If we have to differentiate under the integral, we should make sure our code is differentiable (smoothing) Vibrato MC: combines pathwise derivative and likelihood ratio methods Other approaches possible: see Capriotti and references, Liu and Hong and references Many simple methods will likely misbehave for second order


13

Vibrato Monte Carlo

𝑡𝑖−2

𝑡𝑖−1

𝑋𝑖−2

𝑋𝑖−1

𝑡𝑖

𝑡𝑖+1

𝑡𝑖+2

𝑋𝑖+1

𝑋𝑖+2

𝑋𝑖


14

Vibrato Monte Carlo

𝑡𝑖−2

𝑡𝑖−1

𝑋𝑖−2

𝑋𝑖−1

𝑡𝑖

𝑡𝑖+1

𝑡𝑖+2

𝑋𝑖+1

𝑋𝑖+2

𝑋𝑖 𝔼[𝑓 (𝑋𝑖 ) ∣ 𝑋𝑖−1 , 𝑋𝑖+1 ] = ∫ 𝑓 (𝑥)𝑝(𝑥 | 𝑋𝑖−1 , 𝑋𝑖+1 ) 𝑑𝑥 = 𝐹(𝑋𝑖−1 , 𝑋𝑖+1 ) 𝜕 log(𝑝(𝑥 | 𝑦, 𝑧)) 𝜕𝐹(𝑦, 𝑧) = ∫ 𝑓 (𝑥) 𝑝(𝑥 | 𝑦, 𝑧) 𝑑𝑥 𝜕𝜃 𝜕𝜃


14

Vibrato Monte Carlo

𝑡𝑖−2

𝑡𝑖−1

𝑋𝑖−2

𝑋𝑖−1

𝑡𝑖

𝑡𝑖+1

𝑡𝑖+2

𝑋𝑖+1

𝑋𝑖+2

𝑋𝑖 𝔼[𝑓 (𝑋𝑖 ) ∣ 𝑋𝑖−1 , 𝑋𝑖+1 ] = ∫ 𝑓 (𝑥)𝑝(𝑥 | 𝑋𝑖−1 , 𝑋𝑖+1 ) 𝑑𝑥 = 𝐹(𝑋𝑖−1 , 𝑋𝑖+1 ) 𝜕 log(𝑝(𝑥 | 𝑦, 𝑧)) 𝜕𝐹(𝑦, 𝑧) = ∫ 𝑓 (𝑥) 𝑝(𝑥 | 𝑦, 𝑧) 𝑑𝑥 𝜕𝜃 𝜕𝜃 Let 𝑝 be density of Euler-Maruyama approximation to 𝑋 over Δ𝑡 𝜕𝔼[𝑓 (𝑋𝑖 ) ∣ 𝑋𝑖−1 , 𝑋𝑖+1 ] 𝜕𝑋 𝜕𝑋 = 𝐹𝜃 + 𝐹𝑦 𝑖−1 + 𝐹𝑧 𝑖+1 𝜕𝜃 𝜕𝜃 𝜕𝜃


14

Second order AAD with dco/c++ and dco/map


15

Proto-CVA Code ▶

▶ ▶

Code based on G2++ model with time-dependent vols, rates and forward spreads Doesn’t compute CVA for vanilla swap, but comes close Mock calibration (CPU) followed by Monte Carlo, followed by final aggregation (CPU)

▶

4,000 sample paths, 875 analytic bond prices and LIBOR fixings

▶

62 inputs to calibration, 244 inputs to Monte Carlo, one output


16

Tools dco/c++ ▶ ▶

▶

Very efficient general purpose tape-based (A)AD Easy to use, full control of memory use, has gobs of features, arbitrary order derivatives (used by many banks in production) CPU only

dco/map ▶ ▶

▶

Specialist tape-free (A)AD tool Less easy to use, typically requires source refactoring, first and second order derivatives only Extremely efficient: developed for GPU, but supports any C++11 platform


17

Hardware and Caveat ▶

NVIDIA P100 in an Intel Core i7 4700K quad core machine

▶

Intel dual socket platform

Neither dco/c++ nor dco/map is fully vectorised on CPU, we’re working to fix this


18

dco/c++ on CPU

Passive Gradient runtime runtime 𝑓𝑎 Serial Monte Carlo Total

220 100 320

575.3 1,300 1,882

Memory (MB)

40

700

2.6x 13x 5.9x

62 × 62 Hessian runtime 𝑓𝑎 57,746 143,326 201,070

4.2x 23x 10x

1,200


19

dco/map on CPU with dco/c++

Passive Gradient runtime runtime 𝑓𝑎 Serial (dco/c++)

220

575.3

2.6x

Monte Carlo (dco/map)

100

308

Total

320

882

Memory (MB)

40

1000

62 × 62 Hessian runtime 𝑓𝑎 57,746

4.2x

2.1x

42,564

1.8x

2.7x

100,310

5.1x

1,860


20

dco/map on P100 with dco/c++ on Core i7

Passive Gradient runtime runtime 𝑓𝑎 Serial (dco/c++, Core i7)

114

655

5.7x

62 × 62 Hessian runtime 𝑓𝑎 34,477

4.9x

Monte Carlo (dco/map, P100)

8.3

33

3x

4,770

5.8x

Total

125

688

5.5x

39,247

5x

Memory (MB)

40

1000

1,860


21

Standard Deviations


22

Standard Errors Below are frequency counts of standard deviations at end of Monte Carlo back propagation (roughly 250 variables) Gradient: 1e-6 3

1e-5 14

1e-4 75

1e-3 55

0.01 28

Hessian:

1e-6 1e-5 1e-4 1e-3 0.01 0.1 1 444 2473 7331 5627 5127 4705 3565

Hessian for 100K paths:

1e-6 1e-5 1e-4 1e-3 0.01 0.1 1396 38149 3897 5795 6128 3427

0.2 195

0.1 38

1 32

10 1

30 0

10 816

30 6

60 8

90 8

120 7

150 0

0.3 183

0.4 154

0.5 120

0.6 88

0.7 49

0.8 2

Long tail persists and tail decay is quite slow High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com

23

Batched Least Squares


24

Batched Least Squares ▶

AMC requires least squares solutions of tall skinny matrices

▶

These hard to parallelise

▶

▶ ▶

▶

NAG has developed batched least squares solver running on GPU Input matrices can be on CPU or GPU Performance is very good! Please come visit the stand for more details And of course we can adjoint it as well


25

2nd Order Sensis: PnL and Hedging Chris Kenyon 19.10.2017

Acknowledgements & Disclaimers

Joint work with Jacques du Toit.

The views expressed in this presentation are the personal views of the speaker and do not necessarily reflect the views or policies of current or previous employers. Not guaranteed fit for any purpose. Use at your own risk.

Chatham House Rules apply to any reporting of presentation contents or comments by the speaker.

(c) C.Kenyon 2017

19.10.2017

2 / 17

Before we start ...

If pricing uses first order sensitivities ... ... then hedging needs second order sensitivities

(c) C.Kenyon 2017

Introduction

19.10.2017

3 / 17

Before we start ...

If pricing uses second order sensitivities ... ... then hedging needs third order sensitivities

(c) C.Kenyon 2017

Introduction

19.10.2017

4 / 17

Introduction

Mathematically, there are theorems based on infinitesimals, and on finite differences Infinitesimals (Taylor’s Theorem) Symbolic derivatives of compact equations using analytic expressions (analytic expression have arbitrary operators) Symbolic derivatives of extended equations (i.e. code, computers have only +, −, ×, / operators, roughly speaking)

Large overlap between these Finite differences (Newton’s Theorem) Small, e.g. for sensis Large, e.g. for stresses

Financially we are interested in effects of market movements

(c) C.Kenyon 2017

Introduction

19.10.2017

5 / 17

Definition (Taylor’s Theorem) let k ∈ N > 0 and f : hk (x) : 7→ s.t.

R R

R 7→ R be k times differentiable at a ∈ R then ∃ f (x) = Pk (x) + hk (x)(x − a)k

where Pk (x) is the k-th order Taylor polynomial Pk (x) = f (a) + f 0 (a)(x − a) +

f (k) (a) f 00 (a) (x − a)2 + . . . + (x − a)k 2! k!

and limx→a hk (x) = 0 Hence we can define a remainder Rk (x) := f (x) − Pk (x) = o(|x − a|k ),

x →a

and if f is k + 1 times differentiable on the open interval with f (k) continuous on the closed interval between a and x then, by the Mean Value Theorem, Rk (x) = (c) C.Kenyon 2017

f (k+1) (v ) (x − a)k+1 , (k + 1)! Introduction

v ∈ [a, x] 19.10.2017

6 / 17

Accuracy

Assuming everything works, then with first order derivatives 00

f (v ) R1 (x) = (x − a)2 , 2 and second

v ∈ [a, x]

000

R2 (x) =

f (v ) (x − a)3 , 6

v ∈ [a, x]

cannot do better than this Finite market moves make the above optimistic

(c) C.Kenyon 2017

Introduction

19.10.2017

7 / 17

Mathematical Limitations

Require f must be analytic (i.e. Taylor series must converge to f ) Non-analytic example where Taylor coefficients are all zero at zero ( 2 e −1/x x >0 f (x) = 0 x ≤0

k times differentiable at a k + 1 times differentiable on the open interval with f (k) continuous on the closed interval between a and x Must be able to get the derivatives

There is a radius of convergence within which the approximation works

(c) C.Kenyon 2017

Introduction

19.10.2017

8 / 17

Financial Limitations Exercise boundaries limit availability of first-order differentiability Trade life-cycle: fixings (e.g. with averaging instruments); resets; coupons; notional payments; maturity; transformation (e.g. swaption to swap) Opaque/illiquid model parameters Self and Counterparty life-cycle: rating transitions; default; regulatory permissions Self-Counterparty: CSA change; SwapAgent® , i.e. CTM-to-STM; collateral change Calibration instrument life-cycle: Futures rolls; index rolls; CDS rolls Significant market dates: FOMC meetings; Central Bank meetings Information releases: inflation publication; employment; etc. Gap events: currency life-cycle (start, end, division = pegs); regulatory changes

(c) C.Kenyon 2017

Introduction

19.10.2017

9 / 17

FRTB, PnL Explain

Assume all life-cycle (trade, entities, calibration instrument) and market dates are already included in the explain First order: all Cross-gamma: highly dependent on correlations base-base, e.g. IR curvature base-base, e.g. IR-FX, IR-CM base-vol, e.g. IR and IR vol vol-vol, e.g. FX smile flattening

Diagonal-gamma used but less commonly (also depends on definition of diagonal- vs cross-)

(c) C.Kenyon 2017

Introduction

19.10.2017

10 / 17

How good are your correlations?

Market-implied correlations similar to other market-implied items (rates, vols) but generally require taking positions in several instruments to hedge Historical correlations change as slowly as the calibration algorithm Few models for stochastic correlation — part of more general Wrong Way Risk problem Confidence interval width feeds into Prudential Valuation capital

Default correlation is challenging to estimate from market or historical data

(c) C.Kenyon 2017

Introduction

19.10.2017

11 / 17

XVA

General purpose efficient approach in (Kenyon and Green 2015) t = 0 CVA, FVA hedging needs Forward derivatives of portfolio Jacobian chain back to calibration instruments Cross-gamma of CR-XX vital to capture market risk

Forward derivatives First order: SIMM; CCP IM; FRTB; FRTB-CVA Second order: FRTB (approximate curvature)

Accuracy requirements? Hedging Compression Incremental trading Allocation

(c) C.Kenyon 2017

Introduction

19.10.2017

12 / 17

MVA: first-order sensis in pricing

SIMM, (ISDA-SIMM-2 2017) Delta-vega approach, i.e. first-order Many papers and presentations on using forward derivatives to calculate SIMM Hedging SIMM requires second-order sensis

Regulatory and CCP methods Generally, historical VaR or Expected Shortfall approach (or moving to this) Direct approach (Green and Kenyon 2015) — main issue is change in the key scenarios (Kenyon and Green 2015; Andreasen 2017), which is a jump risk If approximate CCP IM re-using forward sensis developed for SIMM (suggestion from (Chan 2017)) then need second order sensis

Hedging effects of IM (regulatory or CCP) on option exercise also required (Green and Kenyon 2017), also needs second order derivatives No Market Risk capital on MVA (or FVA)

(c) C.Kenyon 2017

Introduction

19.10.2017

13 / 17

KVA, FRTB-CVA-SA: first-order sensis in pricing

Risk Factors Risk Buckets Delta

IR, INF

FX

Delta, Vega Currency

Delta, Vega Currency (not dom) FX spot

Method Vega

Main IR 3 pieces; INF and other IR 1 Relative Single

Method

Relative

Credit (Cpty) Delta Sectors (e.g. IG) 5 pieces

Relative Single

Absolute NA

Relative

NA

Credit (Exp) Delta, Vega Sectors (e.g. IG) Single per bucket

Equity

Commodities

Delta, Vega Sectors (large cap) Single per bucket

Delta, Vega Group

Absolute Single per bucket Relative

Relative Single per bucket Relative

Relative Single per bucket Relative

Single per bucket

KVA using FRTB-CVA-SA requires first order forward sensitivities of CVA Hedging KVA on FRTB-CVA-SA requires second order

(c) C.Kenyon 2017

Introduction

19.10.2017

14 / 17

FRTB, KVA: second-order sensis in pricing? Is this relevant? Main issue is dealing with future trading to maintain t = 0 Market Risk Capital level. FRTB-IMA Expected Shortfall(97.5%), 10-day plus liquidity modification, calibrated to a period of stress Non-modellable risk factors (NMRF) Default risk charge (DRC)

FRTB-SA Sensitivity based: as FRTB-CVA but more detailed + curvature Default risk charge Residual risk add-on

Some work on KVA pricing (Andreasen 2017), but not hedging or allocation. Generally follow pattern of (Kenyon and Green 2015) One open question is whether suggestion of (Chan 2017) to re-use sensitivities obtained for MVA/SIMM for FRTB-IMA is workable

(c) C.Kenyon 2017

Introduction

19.10.2017

15 / 17

Conclusions Many limitations on practical application of Taylor’s Theorem in financial markets from non-differentiability requiring smoothing — and error bound requires next order so full revaluation often more practical First-order sensis in pricing so second-order for hedging: MVA: SIMM; possibly CCP approximation KVA: FRTB-CVA; FRTB-SA (if bump for curvature)

Second-order sensis in pricing so third-order for hedging KVA: FRTB-SA (if use for curvature)

Other second-order sensi uses PnL explain FRTB PnL explain

Revaluation required: Stress testing FRTB-IMA at t = 0

Unclear whether VaR/ES at t = 0 will be permitted using sensis (delta-gamma-vega) rather than full revaluation going forward (c) C.Kenyon 2017

Introduction

19.10.2017

16 / 17

Andreasen, J. (2017). Tricks and Tactics for FRTB. Global Derivatives. Chan, J. (2017). MVA and Capital Efficiency: Accurate Dynamic SIMM Simulation via AAD. MVA Roundtable (Canary Wharf). Green, A. and C. Kenyon (2015, May). MVA by Replication and Regression. Risk 27, 82–87. Green, A. and C. Kenyon (2017). XVA at the Exercise Boundary. Risk. ISDA-SIMM-2 (2017). ISDA SIMM(tm) Methodology Version 2.0. http://www2.isda.org/functional-areas/wgmr-implementation/. Kenyon, C. and A. Green (2015). Efficient XVA Management: Pricing, Hedging, and Allocation. Risk 28.

(c) C.Kenyon 2017

Bibliography

19.10.2017

17 / 17

(c) C.Kenyon 2017

Bibliography

19.10.2017

17 / 17