howto/DN42-Over-SRv6-L3VPN.md
... ...
@@ -0,0 +1,851 @@
1
+Segment Routing is a source-routing (forwarding path is pre-defined by source node) paradigm based technology, through encoding list of instruction called Segment into packet, the forwarding path of packet is defined.
2
+
3
+# Why Segment Routing
4
+Compared to other routing technologies, Segment Routing offers following advantages:
5
+- Simple: No extra signal protocol (LDP, RSVP-TE) needed, SR route information can be propagated through IGPs (IS-IS, OSPF) with related extensions.
6
+- Scalable: Since intermediate nodes don't need to maintain path status, they only need to execute corresponding instruction in Segment list defined in the packet, the resources can be saved more.
7
+- Traffic Engineering: Segment Routing is based on source-routing paradigm, the forwarding path is programmable according to the intend.
8
+- Extinguish of BGP blackhole: Introduce internal second dataplane (MPLS/SRv6), mapping many thousands of BGP routes to few second dataplane routes, making no-BGP intermediate nodes capable forwarding transit traffic, hence the resource is saved.
9
+
10
+The following is a comparison of some popular routing technologies:
11
+
12
+|Name|Simple|Scalable|Traffic Engineering|Extinguish of BGP blackhole|
13
+| ----- | ----- | ----- | ----- | ----- |
14
+|Plain IPv4/IPv6|Yes|Medium|No|No|
15
+|MPLS and LDP|No|Medium|No|Yes|
16
+|MPLS and RSVP-TE|No|Low|Yes|Yes|
17
+|Segment Routing|Yes|High|Yes|Yes|
18
+
19
+## Why SRv6
20
+~~Because this is meant for SRv6.~~
21
+
22
+Segment Routing can run on MPLS (SR-MPLS) and IPv6 (SRv6) dataplane.
23
+
24
+Segment Routing over MPLS (SR-MPLS) is a kind of Segment Routing technology running on MPLS dataplane, it implements Segment Routing through encoding Segment list into MPLS label stack, no modification made to MPLS itself.
25
+
26
+Segment Routing over IPv6 (SRv6) is a kind of Segment Routing technology running on IPv6 dataplane, it implements Segment Routing through introducing a new type of Routing Header: SR Header (SRH) for SRv6 capability.
27
+
28
+Compared to SR-MPLS, SRv6 is a more popular option, it overcomes SR-MPLS at following scopes:
29
+- Better compatibility: SRv6 can be deployed on edge nodes without deploying SRv6 in whole network domain, deployment can be advanced gradually, the intermediate node will transmit SRv6 packet as normal IPv6 packet, while SR-MPLS deployment touches whole network domain.
30
+- More popular: SRv6 is mentioned more, and Linux has integrated it into the network stack without loading extra kernel module; SR-MPLS relied on MPLS, it requires extra kernal module, it's almost not possible for LXC containers.
31
+- Larger address space: SRv6 is based on IPv6, one single SRv6 Locator can be assigned as a /64 network minimum, it provides 2^64 IPv6 addresses for SR network function, while MPLS the SR-MPLS use supports up to 2^20 address space, it's even a little less than the scale of global IPv4 routing table (over 1 million currently), the extensiblity is limited.
32
+
33
+# 1 Preparation
34
+This article assumes the readers have:
35
+- Basic Linux knowledge
36
+- DN42 network setup and operation knowledge, the reader should have been read through [Getting Started](/howto/Getting-Started) and [Universal Network Requirements](/howto/networksettings) in DN42 Wiki
37
+- Haven't yet set up a large scale DN42 network, or you are dare enough to flip it over
38
+
39
+All node are set up with spec as follow:
40
+- Run Debian Linux
41
+- Use GRE over Wireguard for interconnection between nodes
42
+- Run FRR routing software
43
+- Use IS-IS as IGP
44
+- Have DN42-GT and THIS-AS VRF
45
+- Use DN42-GT for external interconnection
46
+- Announce own network route through THIS-AS VRF
47
+
48
+The DN42 over SRv6 L3VPN deployment shown in this article is implemented on my real DN42 network infrastructure.
49
+
50
+## 1.1 Install FRR
51
+For now, most convenient way to implement SRv6 on Linux is to use FRR, and my infrastructure is operating on FRR all time, too.
52
+
53
+Every distro has its way to install, my nodes are mostly running Debian, the download can be found through [https://deb.frrouting.org](https://deb.frrouting.org).
54
+
55
+If you are using Debian as I do, don't forget to install frr-rpki-rtrlib, it's dependency of RPKI protocol for FRR, ROA filter is a must option for DN42.
56
+
57
+The regular FRR DN42 configuration can be referred from [FRRouting](/howto/frr).
58
+
59
+## 1.2 Kernel Parameter Adjustment
60
+Add on the basis of [Universal Network Requirements](/howto/networksettings):
61
+```
62
+net.ipv6.seg6_flowlabel=1
63
+net.ipv6.conf.all.seg6_enabled=1
64
+net.vrf.strict_mode=1
65
+```
66
+**Caution: **net.vrf.strict_mode is a critical parameter, it determines whether different VRF can share one routing table or not, when its value equals 1, every VRF has to have dedicate routing table. If it's value not equal to 1, the SRv6 IPv4 L3VPN won't work, the related SID routes will be rejected.
67
+
68
+## 1.3 VRF Network Interface and VRF Configuration
69
+net.vrf.strict_mode resets 0 everytime a new VRF adds, to prevent network operation interrupt, please create all VRF could be used at most.
70
+
71
+All command followed are **NOT** persistent, the way to persist configuration is on yours.
72
+
73
+The following command are used to create DN42-GT and THIS-AS VRF with iproute2:
74
+```
75
+ip link add DN42-GT type vrf table 300
76
+ip link add THIS-AS type vrf table 100
77
+ip link set DN42-GT up
78
+ip link set THIS-AS up
79
+```
80
+
81
+IS-IS requires a dummy interface for SRv6 Locator route point to (by default it's sr0):
82
+```
83
+ip link add sr0 type dummy
84
+ip link set sr0 up mtu 65536
85
+```
86
+
87
+IS-IS works on tunnel which carries Ethernet frame (gretap, l2tpv3) or supports OSI encapsulation (gre), so create GRE tunnel through IPv4 address on Wireguard interface:
88
+```
89
+ip link add azj1 type gre local 169.254.24.3 remote 169.254.24.17 nopmtudisc
90
+sysctl -w net.ipv6.conf.azj1.seg6_enabled=1
91
+```
92
+Due to unknown reason, FRR can't send IS-IS traffic through ip6gre tunnel, it seems ip6gre don't support OSI encapsulation, since that, don't use IPv6 for GRE tunnel (but ip6gretap can do since it carries Ethernet frame).
93
+
94
+# 2 FRR Configuration
95
+Have every network node configured as follow, this article take my CAN1 node as example.
96
+
97
+IP & SRv6 Locator Assignment:
98
+- SRv6 Locator:5f00:3947:3::/48
99
+- lo:
100
+ - fdfa:7906:8262:ffff::3/128, for BGP L3VPN session use
101
+ - 5f00:3947:3::1/128, for SRv6 source address use
102
+
103
+## 2.1 daemons configuration
104
+Enable BGP and IS-IS, and RPKI support.
105
+```
106
+# This file tells the frr package which daemons to start.
107
+#
108
+# Sample configurations for these daemons can be found in
109
+# /usr/share/doc/frr/examples/.
110
+#
111
+# ATTENTION:
112
+#
113
+# When activating a daemon for the first time, a config file, even if it is
114
+# empty, has to be present *and* be owned by the user and group "frr", else
115
+# the daemon will not be started by /etc/init.d/frr. The permissions should
116
+# be u=rw,g=r,o=.
117
+# When using "vtysh" such a config file is also needed. It should be owned by
118
+# group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too.
119
+#
120
+# The watchfrr, zebra and staticd daemons are always started.
121
+#
122
+bgpd=yes
123
+ospfd=no
124
+ospf6d=no
125
+ripd=no
126
+ripngd=no
127
+isisd=yes
128
+pimd=no
129
+pim6d=no
130
+ldpd=no
131
+nhrpd=no
132
+eigrpd=no
133
+babeld=no
134
+sharpd=no
135
+pbrd=yes
136
+bfdd=yes
137
+fabricd=no
138
+vrrpd=no
139
+pathd=yes
140
+
141
+#
142
+# If this option is set the /etc/init.d/frr script automatically loads
143
+# the config via "vtysh -b" when the servers are started.
144
+# Check /etc/pam.d/frr if you intend to use "vtysh"!
145
+#
146
+vtysh_enable=yes
147
+zebra_options=" -A 127.0.0.1 -s 90000000"
148
+mgmtd_options=" -A 127.0.0.1"
149
+bgpd_options=" -A 127.0.0.1 -M rpki"
150
+ospfd_options=" -A 127.0.0.1"
151
+ospf6d_options=" -A ::1"
152
+ripd_options=" -A 127.0.0.1"
153
+ripngd_options=" -A ::1"
154
+isisd_options=" -A 127.0.0.1"
155
+pimd_options=" -A 127.0.0.1"
156
+pim6d_options=" -A ::1"
157
+ldpd_options=" -A 127.0.0.1"
158
+nhrpd_options=" -A 127.0.0.1"
159
+eigrpd_options=" -A 127.0.0.1"
160
+babeld_options=" -A 127.0.0.1"
161
+sharpd_options=" -A 127.0.0.1"
162
+pbrd_options=" -A 127.0.0.1"
163
+staticd_options="-A 127.0.0.1"
164
+bfdd_options=" -A 127.0.0.1"
165
+fabricd_options="-A 127.0.0.1"
166
+vrrpd_options=" -A 127.0.0.1"
167
+pathd_options=" -A 127.0.0.1"
168
+
169
+
170
+# If you want to pass a common option to all daemons, you can use the
171
+# "frr_global_options" variable.
172
+#
173
+#frr_global_options=""
174
+
175
+
176
+# The list of daemons to watch is automatically generated by the init script.
177
+# This variable can be used to pass options to watchfrr that will be passed
178
+# prior to the daemon list.
179
+#
180
+# To make watchfrr create/join the specified netns, add the the "--netns"
181
+# option here. It will only have an effect in /etc/frr/<somename>/daemons, and
182
+# you need to start FRR with "/usr/lib/frr/frrinit.sh start <somename>".
183
+#
184
+#watchfrr_options=""
185
+
186
+
187
+# configuration profile
188
+#
189
+#frr_profile="traditional"
190
+#frr_profile="datacenter"
191
+
192
+
193
+# This is the maximum number of FD's that will be available. Upon startup this
194
+# is read by the control files and ulimit is called. Uncomment and use a
195
+# reasonable value for your setup if you are expecting a large number of peers
196
+# in say BGP.
197
+#
198
+#MAX_FDS=1024
199
+
200
+# Uncomment this option if you want to run FRR as a non-root user. Note that
201
+# you should know what you are doing since most of the daemons need root
202
+# to work. This could be useful if you want to run FRR in a container
203
+# for instance.
204
+# FRR_NO_ROOT="yes"
205
+
206
+# For any daemon, you can specify a "wrap" command to start instead of starting
207
+# the daemon directly. This will simply be prepended to the daemon invocation.
208
+# These variables have the form daemon_wrap, where 'daemon' is the name of the
209
+# daemon (the same pattern as the daemon_options variables).
210
+#
211
+# Note that when daemons are started, they are told to daemonize with the `-d`
212
+# option. This has several implications. For one, the init script expects that
213
+# when it invokes a daemon, the invocation returns immediately. If you add a
214
+# wrap command here, it must comply with this expectation and daemonize as
215
+# well, or the init script will never return. Furthermore, because daemons are
216
+# themselves daemonized with -d, you must ensure that your wrapper command is
217
+# capable of following child processes after a fork() if you need it to do so.
218
+#
219
+# If your desired wrapper does not support daemonization, you can wrap it with
220
+# a utility program that daemonizes programs, such as 'daemonize'. An example
221
+# of this might look like:
222
+#
223
+# bgpd_wrap="/usr/bin/daemonize /usr/bin/mywrapper"
224
+#
225
+# This is particularly useful for programs which record processes but lack
226
+# daemonization options, such as perf and rr.
227
+#
228
+# If you wish to wrap all daemons in the same way, you may set the "all_wrap"
229
+# variable.
230
+#
231
+#all_wrap=""
232
+```
233
+## 2.2 Configure Interface IP Address
234
+Edit /etc/frr/frr.conf, add configuration followed:
235
+
236
+lo:
237
+```
238
+!
239
+interface lo
240
+ ipv6 address 5f00:3947:3::1/128
241
+ ipv6 address fdfa:7906:8262:ffff::3/128
242
+ ipv6 router isis 1
243
+ mpls enable
244
+exit
245
+!
246
+```
247
+
248
+## 2.3 Configure SRv6 Locator
249
+IANA has assigned 5f00::/16 for SRv6 SID use currently, I have took 5f00:3947::/32 within as my SRv6 network block, then assign /48 size 5f00:3947:x::/48 for every node's SRv6 Locator, use usid-f3216 for SID assignment format.
250
+
251
+Edit /etc/frr/frr.conf, add configuration followed:
252
+```
253
+!
254
+segment-routing
255
+ srv6
256
+ encapsulation
257
+ source-address 5f00:3947:3::1
258
+ exit
259
+ locators
260
+ locator MAIN
261
+ prefix 5f00:3947:3::/48 block-len 32 node-len 16
262
+ behavior usid
263
+ format usid-f3216
264
+ exit
265
+ !
266
+ exit
267
+ !
268
+ exit
269
+ !
270
+exit
271
+!
272
+```
273
+If you want to use your own DN42 IPv6 as SRv6 Locator, then assign every node /64 size SRv6 Locator, please use usid-f4816 instead.
274
+
275
+## 2.4 Configure IS-IS SRv6
276
+After SRv6 Locator configuration, the IS-IS configuration is also required for announcing SRv6 Locator route.
277
+
278
+As why IS-IS, because IS-IS is only IGP supports SRv6 in FRR for now.
279
+
280
+IS-IS uses NET (Network Entity Title) in CLNS format to identify node, it's length variable, consisted mainly in two parts:
281
+- Area ID (1~13 bytes)
282
+- System ID (7 bytes)
283
+ - Station ID (6 bytes)
284
+ - Selector (1 byte)
285
+
286
+The Area ID format has following popular varient:
287
+- Single byte Area Address
288
+ - 00
289
+- AFI (1 byte) + Area (2 bytes) AFI (1 byte) + Area (2 bytes)
290
+ - 49.0000
291
+- AFI (1 byte) + Domain (2 bytes) + Area (2 bytes)
292
+ - 49.0000.0000
293
+
294
+CAN1 has assined NET 49.0000.1720.2020.9096.00, add follow into /etc/frr/frr.conf:
295
+```
296
+!
297
+router isis 1
298
+ is-type level-1
299
+ net 49.0000.1720.2020.9096.00
300
+ lsp-mtu 1277
301
+ segment-routing on
302
+ segment-routing srv6
303
+ locator MAIN
304
+ exit
305
+exit
306
+!
307
+```
308
+
309
+## 2.5 Configure BGP SRv6
310
+After configuration of SRv6 route announcement through IGP, BGP SRv6 Locator configuration is also needed, BGP L3VPN will use this SRv6 Locator to generate then bind corresponding SID for VRF.
311
+
312
+Add follow to BGP configuration block in /etc/frr/frr.conf:
313
+```
314
+ !
315
+ segment-routing srv6
316
+ locator MAIN
317
+ exit
318
+ !
319
+```
320
+If you don't know where to insert, you may refer complete configuration in later part.
321
+
322
+## 2.6 Configure SRv6 L3VPN
323
+Setup BGP L3VPN Peer in default VRF:
324
+```
325
+router bgp 4242423947
326
+ bgp router-id 172.20.209.96
327
+ no bgp default ipv4-unicast
328
+ neighbor ibgp peer-group
329
+ neighbor ibgp remote-as 4242423947
330
+ neighbor ibgp update-source lo
331
+ neighbor ibgp capability extended-nexthop
332
+ neighbor fdfa:7906:8262:ffff::1 peer-group ibgp
333
+ neighbor fdfa:7906:8262:ffff::1 description hkg1
334
+ neighbor fdfa:7906:8262:ffff::6 peer-group ibgp
335
+ neighbor fdfa:7906:8262:ffff::6 description fra1
336
+```
337
+Activate BGP L3VPN Peer:
338
+```
339
+address-family ipv4 vpn
340
+ neighbor ibgp activate
341
+ exit-address-family
342
+```
343
+```
344
+address-family ipv6 vpn
345
+ neighbor ibgp activate
346
+ exit-address-family
347
+```
348
+If you don't know where to insert, you may refer complete configuration in later part.
349
+
350
+Configure VPN import/export, RD, RT import/export and SID binding for IPv4 Unicast and IPv6 Unicast address family in **corresponding VRF**:
351
+```
352
+ address-family ipv4 unicast
353
+ sid vpn export auto
354
+ rd vpn export 172.20.209.96:100
355
+ rt vpn import 4242423947:100 4242423947:300 4242423947:301 4242423947:500
356
+ rt vpn export 4242423947:100
357
+ export vpn
358
+ import vpn
359
+ exit-address-family
360
+ !
361
+ address-family ipv6 unicast
362
+ sid vpn export auto
363
+ rd vpn export 172.20.209.96:100
364
+ rt vpn import 4242423947:100 4242423947:300 4242423947:301 4242423947:500
365
+ rt vpn export 4242423947:100
366
+ export vpn
367
+ import vpn
368
+ exit-address-family
369
+```
370
+If you don't know where to insert, you may refer complete configuration in later part.
371
+
372
+## 2.7 CAN1 Full FRR Configuration
373
+I provide full FRR configuration of CAN1 without redundant route-map and eBGP peer for reference, the VRF RPKI and BFD configuration is also included.
374
+```
375
+frr version 10.6.1
376
+frr defaults traditional
377
+hostname SERNET-CAN1
378
+log syslog informational
379
+service integrated-vtysh-config
380
+!
381
+ip prefix-list dn42-subnet seq 1100 permit 172.20.0.0/14 le 32
382
+ip prefix-list dn42-subnet seq 2001 permit 10.100.0.0/14 le 32
383
+ip prefix-list dn42-subnet seq 2002 permit 10.127.0.0/16 le 32
384
+ip prefix-list dn42-subnet seq 2003 permit 10.0.0.0/8 ge 15 le 24
385
+ip prefix-list dn42-subnet seq 3001 permit 172.31.0.0/16 le 32
386
+ip prefix-list dn42-subnet seq 9999 deny 0.0.0.0/0 le 32
387
+ip prefix-list global seq 1 permit 172.20.209.64/26
388
+ip prefix-list local seq 2 permit 172.20.209.120/29
389
+ip prefix-list local seq 3 permit 172.20.209.96/28
390
+ip prefix-list vrf_local seq 1 permit 169.254.23.0/24 le 32
391
+!
392
+ipv6 prefix-list dn42-subnet seq 1 permit fd00::/8 ge 44 le 64
393
+ipv6 prefix-list dn42-subnet seq 65535 deny ::/0 le 128
394
+ipv6 prefix-list global seq 1 permit fdfa:7906:8262::/48
395
+!
396
+route-map inbound permit 10
397
+ on-match next
398
+ set local-preference 100
399
+exit
400
+!
401
+route-map inbound permit 20
402
+ match as-path adj
403
+ on-match next
404
+ set local-preference +150
405
+exit
406
+!
407
+route-map inbound permit 21
408
+ match community region
409
+ on-match next
410
+ set local-preference +110
411
+exit
412
+!
413
+route-map inbound permit 22
414
+ match community country
415
+ set local-preference +110
416
+exit
417
+!
418
+route-map inbound permit 65535
419
+ call rpki
420
+exit
421
+!
422
+route-map local-as-only permit 10
423
+ match as-path self
424
+exit
425
+!
426
+route-map outbound permit 10
427
+ call set-community
428
+ match as-path self
429
+ match rpki valid
430
+exit
431
+!
432
+route-map outbound permit 20
433
+ call rpki
434
+exit
435
+!
436
+route-map redistribute-connected permit 10
437
+ match ip address prefix-list dn42-subnet
438
+exit
439
+!
440
+route-map redistribute-connected permit 20
441
+ match ipv6 address prefix-list dn42-subnet
442
+exit
443
+!
444
+route-map rpki permit 10
445
+ match rpki valid
446
+exit
447
+!
448
+route-map rpki permit 20
449
+ match rpki notfound
450
+ on-match goto 40
451
+exit
452
+!
453
+route-map rpki deny 30
454
+ match rpki invalid
455
+exit
456
+!
457
+route-map rpki permit 40
458
+ match ip address prefix-list dn42-subnet
459
+exit
460
+!
461
+route-map rpki permit 41
462
+ match ipv6 address prefix-list dn42-subnet
463
+exit
464
+!
465
+route-map set-community permit 10
466
+ set community 64511:52 64511:1156
467
+exit
468
+!
469
+ip route 172.20.0.0/14 dn42-service nexthop-vrf THIS-AS
470
+ip route 172.31.0.0/16 dn42-service nexthop-vrf THIS-AS
471
+ip route 172.16.135.0/24 dn42-service nexthop-vrf THIS-AS
472
+ipv6 route fd00::/8 dn42-service nexthop-vrf THIS-AS
473
+ipv6 route fdfa:7906:8262:c530::/64 dn42-service nexthop-vrf THIS-AS
474
+ip router-id 172.20.209.96
475
+!
476
+vrf DN42-GT
477
+ ipv6 route fdfa:7906:8262:c530::/64 dn42-service
478
+ rpki
479
+ rpki cache tcp 169.254.23.1 8082 preference 10
480
+ exit
481
+exit-vrf
482
+!
483
+vrf THIS-AS
484
+ ip route 0.0.0.0/0 ens5 nexthop-vrf default
485
+ ip route 10.0.0.0/8 Null0
486
+ ip route 172.20.0.0/14 Null0
487
+ ip route 10.127.217.0/24 Null0
488
+ ip route 172.20.209.64/26 Null0
489
+ ipv6 route fd00::/8 Null0
490
+ ipv6 route fdfa:7906:8262::/48 Null0
491
+ ipv6 route fdfa:7906:8262:cf00::/56 Null0
492
+ ipv6 route fdfa:7906:8262:c530::/64 dn42-service
493
+ ipv6 route fdfa:7906:8262:cf00::/64 access-hawkins
494
+exit-vrf
495
+!
496
+vrf DN42-PT
497
+exit-vrf
498
+!
499
+vrf NEO-HAWKINS
500
+exit-vrf
501
+!
502
+interface DN42-GT
503
+ ip address 169.254.23.1/32
504
+exit
505
+!
506
+interface THIS-AS
507
+ ip address 10.127.217.66/32
508
+ ip address 172.20.209.96/32
509
+ ipv6 address fdfa:7906:8262:f::3/128
510
+ ipv6 address fdfa:7906:8262:ffff::3/128
511
+exit
512
+!
513
+interface azj1
514
+ ipv6 router isis 1
515
+ isis bfd
516
+ isis bfd profile internet
517
+ isis fast-reroute lfa
518
+ isis network point-to-point
519
+ mpls enable
520
+exit
521
+!
522
+interface ctu1
523
+ ipv6 router isis 1
524
+ isis bfd
525
+ isis bfd profile internet
526
+ isis fast-reroute lfa
527
+ isis network point-to-point
528
+ mpls enable
529
+ link-params
530
+ exit-link-params
531
+exit
532
+!
533
+interface dn42-service
534
+ pbr-policy to-dn42
535
+exit
536
+!
537
+interface fra1
538
+ ipv6 router isis 1
539
+ isis bfd
540
+ isis bfd profile internet
541
+ isis fast-reroute lfa
542
+ isis metric 20
543
+ isis network point-to-point
544
+ mpls enable
545
+ link-params
546
+ exit-link-params
547
+exit
548
+!
549
+interface hkg1
550
+ ipv6 router isis 1
551
+ isis bfd
552
+ isis bfd profile internet
553
+ isis fast-reroute lfa
554
+ isis metric 4
555
+ isis network point-to-point
556
+ mpls enable
557
+ link-params
558
+ exit-link-params
559
+exit
560
+!
561
+interface lo
562
+ ipv6 address 5f00:3947:3::1/128
563
+ ipv6 address fdfa:7906:8262:ffff::3/128
564
+ ipv6 router isis 1
565
+ mpls enable
566
+exit
567
+!
568
+interface sjc1
569
+ ipv6 router isis 1
570
+ isis bfd
571
+ isis bfd profile internet
572
+ isis fast-reroute lfa
573
+ isis metric 20
574
+ isis network point-to-point
575
+ mpls enable
576
+ link-params
577
+ exit-link-params
578
+exit
579
+!
580
+interface tyo1
581
+ ipv6 router isis 1
582
+ isis bfd
583
+ isis bfd profile internet
584
+ isis fast-reroute lfa
585
+ isis metric 15
586
+ isis network point-to-point
587
+ mpls enable
588
+ link-params
589
+ exit-link-params
590
+exit
591
+!
592
+interface wds1
593
+ ipv6 router isis 1
594
+ isis bfd
595
+ isis bfd profile internet
596
+ isis fast-reroute lfa
597
+ isis network point-to-point
598
+ mpls enable
599
+ link-params
600
+ exit-link-params
601
+exit
602
+!
603
+router bgp 4242423947
604
+ bgp router-id 172.20.209.96
605
+ no bgp default ipv4-unicast
606
+ neighbor ibgp peer-group
607
+ neighbor ibgp remote-as 4242423947
608
+ neighbor ibgp update-source lo
609
+ neighbor ibgp capability extended-nexthop
610
+ neighbor fdfa:7906:8262:ffff::1 peer-group ibgp
611
+ neighbor fdfa:7906:8262:ffff::1 description hkg1
612
+ neighbor fdfa:7906:8262:ffff::6 peer-group ibgp
613
+ neighbor fdfa:7906:8262:ffff::6 description fra1
614
+ !
615
+ segment-routing srv6
616
+ locator MAIN
617
+ no srv6-only
618
+ exit
619
+ !
620
+ address-family ipv4 unicast
621
+ label vpn export auto
622
+ sid vpn export auto
623
+ rd vpn export 172.20.209.96:0
624
+ rt vpn export 4242423947:100
625
+ export vpn
626
+ exit-address-family
627
+ !
628
+ address-family ipv4 vpn
629
+ neighbor ibgp activate
630
+ exit-address-family
631
+ !
632
+ address-family ipv6 unicast
633
+ label vpn export auto
634
+ sid vpn export auto
635
+ rd vpn export 172.20.209.96:0
636
+ rt vpn export 4242423947:100
637
+ export vpn
638
+ exit-address-family
639
+ !
640
+ address-family ipv6 vpn
641
+ neighbor ibgp activate
642
+ exit-address-family
643
+exit
644
+!
645
+router bgp 4242423947 vrf DN42-GT
646
+ bgp router-id 172.20.209.96
647
+ no bgp enforce-first-as
648
+ no bgp default ipv4-unicast
649
+ no bgp network import-check
650
+ !
651
+ address-family ipv4 unicast
652
+ neighbor ebgp-DN42-GT activate
653
+ neighbor ebgp-DN42-GT route-map inbound in
654
+ neighbor ebgp-DN42-GT route-map outbound out
655
+ neighbor ebgp4-DN42-GT activate
656
+ neighbor ebgp4-DN42-GT route-map inbound in
657
+ neighbor ebgp4-DN42-GT route-map outbound out
658
+ neighbor ebgp6-DN42-GT activate
659
+ sid vpn export auto
660
+ rd vpn export 172.20.209.96:300
661
+ rt vpn import 4242423947:100 4242423947:300 4242423947:301 4242423947:500
662
+ rt vpn export 4242423947:300
663
+ export vpn
664
+ import vpn
665
+ exit-address-family
666
+ !
667
+ address-family ipv6 unicast
668
+ neighbor ebgp-DN42-GT activate
669
+ neighbor ebgp-DN42-GT route-map inbound in
670
+ neighbor ebgp-DN42-GT route-map outbound out
671
+ neighbor ebgp6-DN42-GT activate
672
+ neighbor ebgp6-DN42-GT route-map inbound in
673
+ neighbor ebgp6-DN42-GT route-map outbound out
674
+ sid vpn export auto
675
+ rd vpn export 172.20.209.96:300
676
+ rt vpn import 4242423947:100 4242423947:300 4242423947:301 4242423947:500
677
+ rt vpn export 4242423947:300
678
+ export vpn
679
+ import vpn
680
+ exit-address-family
681
+exit
682
+!
683
+router bgp 4242423947 vrf THIS-AS
684
+ no bgp default ipv4-unicast
685
+ neighbor 172.16.135.2 remote-as 4242423947
686
+ neighbor 172.16.135.2 description exabgp
687
+ neighbor fdfa:7906:8262:c530::2 remote-as 4242423947
688
+ neighbor fdfa:7906:8262:c530::2 description exabgp
689
+ !
690
+ address-family ipv4 unicast
691
+ network 10.127.217.0/24
692
+ network 10.127.217.2/32
693
+ network 10.127.217.66/32
694
+ network 172.16.128.0/24
695
+ network 172.20.209.64/26
696
+ network 172.20.209.96/32
697
+ redistribute connected route-map redistribute-connected
698
+ neighbor 172.16.135.2 activate
699
+ sid vpn export auto
700
+ rd vpn export 172.20.209.96:100
701
+ rt vpn import 4242423947:100 4242423947:300 4242423947:301 4242423947:500
702
+ rt vpn export 4242423947:100
703
+ export vpn
704
+ import vpn
705
+ exit-address-family
706
+ !
707
+ address-family ipv6 unicast
708
+ network fdfa:7906:8262::/48
709
+ network fdfa:7906:8262:f::3/128
710
+ network fdfa:7906:8262:c500::/64
711
+ network fdfa:7906:8262:c530::/64
712
+ network fdfa:7906:8262:cf00::/56
713
+ redistribute connected route-map redistribute-connected
714
+ neighbor fdfa:7906:8262:c530::2 activate
715
+ sid vpn export auto
716
+ rd vpn export 172.20.209.96:100
717
+ rt vpn import 4242423947:100 4242423947:300 4242423947:301 4242423947:500
718
+ rt vpn export 4242423947:100
719
+ export vpn
720
+ import vpn
721
+ exit-address-family
722
+exit
723
+!
724
+router isis 1
725
+ is-type level-1
726
+ net 49.0000.1720.2020.9096.00
727
+ lsp-mtu 1277
728
+ segment-routing on
729
+ segment-routing srv6
730
+ locator MAIN
731
+ exit
732
+exit
733
+!
734
+bgp as-path access-list adj seq 5 permit ^[0-9]*$
735
+bgp as-path access-list always-send seq 1 permit ^$
736
+bgp as-path access-list self seq 5 permit ^$
737
+!
738
+bgp community-list standard country seq 1 permit 64511:1156
739
+bgp community-list standard region seq 1 permit 64511:52
740
+bgp extcommunity-list standard no-import-rt seq 1 permit rt 4242423947:0
741
+bgp extcommunity-list standard premium-rt seq 1 permit rt 4242423947:100
742
+bgp extcommunity-list standard premium-rt seq 2 permit rt 4242423947:301
743
+!
744
+pbr-map to-dn42 seq 1
745
+ match dst-ip fd00::/8
746
+ set vrf THIS-AS
747
+exit
748
+!
749
+pbr-map to-dn42 seq 2
750
+ match dst-ip 10.0.0.0/8
751
+ set vrf THIS-AS
752
+exit
753
+!
754
+pbr-map to-dn42 seq 3
755
+ match dst-ip 172.20.0.0/14
756
+ set vrf THIS-AS
757
+exit
758
+!
759
+pbr-map to-dn42 seq 4
760
+ match dst-ip 172.31.0.0/16
761
+ set vrf THIS-AS
762
+exit
763
+!
764
+pbr-map to-dn42 seq 5
765
+ match dst-ip 172.16.128.0/24
766
+ set vrf THIS-AS
767
+exit
768
+!
769
+segment-routing
770
+ srv6
771
+ encapsulation
772
+ source-address 5f00:3947:3::1
773
+ exit
774
+ locators
775
+ locator MAIN
776
+ prefix 5f00:3947:3::/48 block-len 32 node-len 16
777
+ behavior usid
778
+ format usid-f3216
779
+ exit
780
+ !
781
+ exit
782
+ !
783
+ exit
784
+ !
785
+ traffic-eng
786
+ exit
787
+exit
788
+!
789
+bfd
790
+ profile internet
791
+ detect-multiplier 5
792
+ transmit-interval 1000
793
+ receive-interval 1000
794
+ exit
795
+ !
796
+exit
797
+!
798
+```
799
+
800
+# 3 Verification
801
+The verification next is proceeded on a PC connected to DN42 network through CAN1.
802
+## 3.1 Packet Capture
803
+Use Wireshark SSH remote capture for link hkg1 on CAN1 which connects to HKG1.
804
+
805
+To allow readers view SRv6 capture more intuitive, PCAP filter is configured:
806
+![pcap filter](https://blog.sherpherd.net/img/%E5%B1%8F%E5%B9%95%E6%88%AA%E5%9B%BE%202026-07-30%20182844.png)
807
+
808
+Then packet capture can be initiated.
809
+## 3.2 Ping internal address
810
+172.20.209.64 is configured on HKG1, hence it must go through link hkg1:
811
+```
812
+PS C:\Users\haksr> ping 172.20.209.64
813
+
814
+Pinging 172.20.209.64 with 32 bytes of data:
815
+Reply from 172.20.209.64: bytes=32 time=22ms TTL=63
816
+Reply from 172.20.209.64: bytes=32 time=24ms TTL=63
817
+Reply from 172.20.209.64: bytes=32 time=24ms TTL=63
818
+Reply from 172.20.209.64: bytes=32 time=24ms TTL=63
819
+
820
+Ping statistics for 172.20.209.64:
821
+ Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
822
+Approximate round trip times in milli-seconds:
823
+ Minimum = 22ms, Maximum = 24ms, Average = 23ms
824
+```
825
+## 3.3 Ping external address with transit node
826
+Traffic to wiki.dn42 (172.23.0.80) go through hkg1 too:
827
+```
828
+Pinging 172.23.0.80 with 32 bytes of data:
829
+Reply from 172.23.0.80: bytes=32 time=29ms TTL=63
830
+Reply from 172.23.0.80: bytes=32 time=28ms TTL=63
831
+Reply from 172.23.0.80: bytes=32 time=32ms TTL=63
832
+Reply from 172.23.0.80: bytes=32 time=26ms TTL=63
833
+
834
+Ping statistics for 172.23.0.80:
835
+ Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
836
+Approximate round trip times in milli-seconds:
837
+ Minimum = 26ms, Maximum = 32ms, Average = 28ms
838
+```
839
+## 3.4 Access wiki.dn42
840
+![wiki.dn42](https://blog.sherpherd.net/img/%E5%B1%8F%E5%B9%95%E6%88%AA%E5%9B%BE%202026-07-30%20183922.png)
841
+## 3.5 View packet capture
842
+![packet capture](https://blog.sherpherd.net/img/%E5%B1%8F%E5%B9%95%E6%88%AA%E5%9B%BE%202026-07-30%20184327.png)
843
+
844
+[srv6_2026-07-30-1838](https://blog.sherpherd.net/blob/srv6_2026-07-30-1838.pcapng)
845
+
846
+# 4 Known limits
847
+## 4.1 NAT
848
+When destination route has SRv6 encapsulation, the traffic won't trigger SNAT rule in netfilter, instead, it encapsulates into SRv6 traffic directly then send out.
849
+
850
+## 4.2 Router IPv6 address inaccessible
851
+Due to unknown reason, the call path of SRv6 decapsulation is different for IPv4 and IPv6, the IPv4 return traffic can lookup destined VRF normally, while IPv6 don't. However, this don't affect transit traffic, only traffic sources or destined to the router is affected.