Jio VoWiFi issue – It’s always DNS!

tl;dr If Jio VoWiFi isn’t working for you, set a different DNS resolver on the phone. While I am a big proponent of running your own resolver in the network, you could test by using open resolvers. The issue doesn’t seem to be impacting everyone and only a subset of users.

To begin with, there are multiple things broken in the authoritative name servers ns1.vowifi.jio.com. and ns2.vowifi.jio.com. of vowifi.jio.com which I’ll cover a bit later.

I came across reports ( See here & here ) of Jio VoWiFi not working for many and while the reports were sketchy, I decided to test this myself.

Below is a snippet from a log file of a dns query to vowifi.jio.com from my phone(192.168.1.137) to a recursive resolver(Unbound) which I run in my network,

May 28 15:54:35 root unbound: [1300:0] info: 192.168.1.137 vowifi.jio.com. A IN

Ideally, the domain is standardised & is made up of Mobile Network Code(MNC) and Mobile Country Code(MCC). For example – in the case of Airtel VoWiFi, the domain name that I see hitting my Unbound resolver is epdg.epc.mnc045.mcc404.pub.3gppnetwork.org. where MNC – 045 and MCC – 404 which signifies Airtel – Karnataka region.

However, oddly enough, Reliance Jio seems to be using vowifi.jio.com. Having said that, the standardised domain name works as well. For example – epdg.epc.mnc861.mcc405.pub.3gppnetwork.org. resolves to 49.44.59.36 and 49.44.59.38

Below is the dns resolution entire delegation chain. From my home network, I can see that the vowifi.jio.com resolves to 49.44.59.38 and 49.44.59.36

.	518400	IN	NS	a.root-servers.net.
.	518400	IN	NS	b.root-servers.net.
.	518400	IN	NS	c.root-servers.net.
.	518400	IN	NS	d.root-servers.net.
.	518400	IN	NS	e.root-servers.net.
.	518400	IN	NS	f.root-servers.net.
.	518400	IN	NS	g.root-servers.net.
.	518400	IN	NS	h.root-servers.net.
.	518400	IN	NS	i.root-servers.net.
.	518400	IN	NS	j.root-servers.net.
.	518400	IN	NS	k.root-servers.net.
.	518400	IN	NS	l.root-servers.net.
.	518400	IN	NS	m.root-servers.net.
com.	172800	IN	NS	a.gtld-servers.net.
com.	172800	IN	NS	l.gtld-servers.net.
com.	172800	IN	NS	c.gtld-servers.net.
com.	172800	IN	NS	h.gtld-servers.net.
com.	172800	IN	NS	e.gtld-servers.net.
com.	172800	IN	NS	d.gtld-servers.net.
com.	172800	IN	NS	i.gtld-servers.net.
com.	172800	IN	NS	f.gtld-servers.net.
com.	172800	IN	NS	m.gtld-servers.net.
com.	172800	IN	NS	j.gtld-servers.net.
com.	172800	IN	NS	g.gtld-servers.net.
com.	172800	IN	NS	k.gtld-servers.net.
com.	172800	IN	NS	b.gtld-servers.net.
jio.com.	172800	IN	NS	ns1.jio.com.
jio.com.	172800	IN	NS	ns2.jio.com.
jio.com.	172800	IN	NS	ns3.jio.com.
jio.com.	172800	IN	NS	ns4.jio.com.
vowifi.jio.com.	3600	IN	NS	ns1.vowifi.jio.com.
vowifi.jio.com.	3600	IN	NS	ns2.vowifi.jio.com.
vowifi.jio.com.	5	IN	A	49.44.59.38
vowifi.jio.com.	5	IN	A	49.44.59.36

At this point, I confirmed that VoWiFi on Jio works by putting the phone on Airplane mode while remain connected to WiFi. A ~22 minute call worked flawlessly.

To confirm that vowifi.jio.com was indeed the domain name that needs to resolve for VoWiFi to work on Jio, I configured an entry for vowifi.jio.com to return a NXDOMAIN answer in my DNS RPZ aka DNS Firewall in Unbound.

With that configured, any DNS query for vowifi.jio.com from any device in the network will be meted out with a NXDOMAIN answer. Below is a snippet from the Unbound log confirming the RPZ rule applied.

May 28 17:31:50 root unbound: [1191:0] info: 192.168.0.137 vowifi.jio.com. A IN
May 28 17:31:50 root unbound: [1191:0] info: RPZ applied [custom block to test vowifi] vowifi.jio.com. nxdomain 192.168.0.137@64521 vowifi.jio.com. A IN
;; ->>HEADER<<- opcode: QUERY, rcode: NXDOMAIN, id: 14747
;; flags: qr aa rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; vowifi.jio.com.	IN	A

;; ANSWER SECTION:

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 136 msec
;; SERVER: 192.168.0.250
;; WHEN: Thu May 28 18:03:42 2020
;; MSG SIZE  rcvd: 32

And VoWiFi(Jio) stops working.

Image of iPhone while making a call using VoWiFi

You can refer to the MNC and MCC codes list on Wikipedia – Mobile Network Codes in ITU region 4xx (Asia)

In the context of VoWiFi, the other noticeable problems with DNS infrastructure of Jio –

  1. A/AAAA records for ns1.vowifi.jio.com, ns2.vowifi.jio.com are missing
  2. ns1.vowifi.jio.com(49.44.59.6), ns2.vowifi.jio.com(49.44.59.7) don’t respond to queries over TCP

The other interesting thing that is worth observing is that when you try resolving vowifi.jio.com from outside India or use a DNS resolver which is perhaps not geographically located within India, the authoritative name servers ns1.vowifi.jio.com(49.44.59.6), ns2.vowifi.jio.com(49.44.59.7) give out a different set of IP addresses – 49.45.63.1, 49.45.63.2

; <<>> DiG 9.16.3 <<>> @127.0.0.1 vowifi.jio.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13728
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;vowifi.jio.com.			IN	A

;; ANSWER SECTION:
vowifi.jio.com.		4	IN	A	49.45.63.1
vowifi.jio.com.		4	IN	A	49.45.63.2

;; Query time: 352 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat May 30 06:23:23 IST 2020
;; MSG SIZE  rcvd: 75

@varkey at IBF pointed out the OpenDNS Cache Check website which also seems to confirm it.

To confirm this hypothesis, I decided to utilise the RIPE Atlas probes to run a measurement. If you’re unaware of the RIPE Atlas project, check an earlier post – Host a RIPE Atlas software probe in your network.

And the results of the measurement are interesting. Out of the 75 probes which participated in the measurement, there were many probes which received the response 49.45.63.1 & 49.45.63.2 to the DNS query to vowifi.jio.com

ASNAS NameDNS Response 1DNS Response 2Resolver IP address
4758NICNET-VSNL-BOARDER-AP National Informatics Centre, IN’49.45.63.249.45.63.1164.100.3.1
4758NICNET-VSNL-BOARDER-AP National Informatics Centre, IN’49.45.63.149.45.63.2164.100.3.1
24186RAILTEL-AS-IN RailTel Corporation of India Ltd., Internet Service Provider, New Delhi, IN’49.44.59.3649.44.59.38127.0.0.1
14061DIGITALOCEAN-ASN, US’49.44.59.3649.44.59.38127.0.0.1
18209BEAMTELE-AS-AP Atria Convergence Technologies pvt ltd, IN’49.44.59.3849.44.59.36202.53.8.8
18209BEAMTELE-AS-AP Atria Convergence Technologies pvt ltd, IN’49.44.59.3849.44.59.3649.207.46.6
135190UBERCORE-AS Ubercore Data Labs Private Limited, IN’49.45.63.149.45.63.2192.168.1.200
135817ESTOB-AS-AP Esto Broadband Private Limited, IN’49.45.63.149.45.63.28.8.8.8
18207YOU-INDIA-AP YOU Broadband & Cable India Ltd., IN’49.45.63.149.45.63.28.8.8.8
134316WORLD-AS World Star Communication, IN’49.44.59.3849.44.59.361.1.1.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3849.44.59.36192.168.0.1
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.44.59.3649.44.59.3810.98.0.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3849.44.59.36192.168.1.1
17625BLAZENET-IN-AP BlazeNet_s Network, IN’49.44.59.3849.44.59.36202.131.104.2
133661NETPLUS-AS Netplus Broadband Services Private Limited, IN’49.45.63.149.45.63.2192.168.1.2
133982EXCITEL-AS-IN Excitel Broadband Private Limited, IN’49.44.59.3649.44.59.38192.168.1.1
133318MAXTECHA-AS Maxtech, IN’None49.44.59.38192.168.1.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3649.44.59.381.1.1.1
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.45.63.249.45.63.1fd00:1:2:3::1
131442DIGITALNETWORK-IN Digital Network Associates Pvt Ltd, IN’49.45.63.249.45.63.1192.168.10.1
135260FOURTY2COMM-AS 42 Communications Pvt. Ltd., IN’None49.45.63.1192.168.10.1
9430STPI-NOIDA Software Technology Parks of India,Block-IV, IN’49.44.59.3649.44.59.38192.168.1.1
132933CTPLAND-AS CharotarTelelink Pvt Ltd, IN’49.45.63.149.45.63.2192.168.1.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3849.44.59.3645.90.28.112
17488HATHWAY-NET-AP Hathway IP Over Cable Internet, IN’49.44.59.3649.44.59.38202.88.152.8
55824NKN-CORE-NW NKN Core Network, IN’49.45.63.149.45.63.2200.200.200.7
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’None49.45.63.2200.200.200.7
18196SEVENSTAR-AS Seven Star Internet Service Provider, IN’49.45.63.149.45.63.28.8.8.8
9829BSNL-NIB National Internet Backbone, IN’49.45.63.149.45.63.22001:4860:4860::8888
134053EXPL-AS-IN ETHERNET XPRESS PVT. LTD., IN’49.44.59.3849.44.59.36fda9:ded9:2bc5::1
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.45.63.249.45.63.1192.168.1.1
9829BSNL-NIB National Internet Backbone, IN’49.44.59.3649.44.59.38192.168.1.1
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.45.63.149.45.63.210.0.0.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3849.44.59.36202.56.215.55
55824NKN-CORE-NW NKN Core Network, IN’49.44.59.3849.44.59.36192.168.1.7
17747SITINETWORS-IN-AP SITI NETWORKS LIMITED, IN’49.45.63.149.45.63.2172.22.146.1
134053EXPL-AS-IN ETHERNET XPRESS PVT. LTD., IN’49.44.59.3649.44.59.3845.116.0.238
24186RAILTEL-AS-IN RailTel Corporation of India Ltd., Internet Service Provider, New Delhi, IN’49.44.59.3649.44.59.3810.0.7.253
23860ALLIANCE-GATEWAY-AS-AP Alliance Broadband Services Pvt. Ltd., IN’49.44.59.3649.44.59.38203.171.240.10
132215POWERGRID-IN Power Grid Corporation of India Limited, IN’49.45.63.149.45.63.210.0.0.1
9498BBIL-AP BHARTI Airtel Ltd., IN’49.45.63.249.45.63.1192.168.0.44
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.44.59.3849.44.59.36192.168.1.1
17813MTNL-AP Mahanagar Telephone Nigam Limited, IN’49.44.59.3649.44.59.3859.185.3.10
45528TIKONAIN-AS Tikona Infinet Ltd., IN’49.44.59.3849.44.59.36192.168.1.1
134325JETSPOTNETWORKSPVTLTD-AS JETSPOTNETWORKS PVT LTD, IN’49.44.59.3649.44.59.38192.168.1.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3649.44.59.38192.168.1.1
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.44.59.3849.44.59.36202.83.21.43
17747SITINETWORS-IN-AP SITI NETWORKS LIMITED, IN’49.44.59.3649.44.59.381.1.1.1
9829BSNL-NIB National Internet Backbone, IN’49.44.59.3849.44.59.36192.168.1.1
134249MARGONW-AS Margo Networks Pvt Ltd, IN’49.44.59.3649.44.59.38172.28.242.252
12222AKAMAI, US’49.45.63.249.45.63.123.216.52.9
55836RELIANCEJIO-IN Reliance Jio Infocomm Limited, IN’49.44.59.3849.44.59.36192.168.29.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3649.44.59.38192.168.1.1
136336TICFIBER-AS Thamizhaga Internet Communications Private Limited, IN’49.44.59.3849.44.59.36192.168.0.1
9829BSNL-NIB National Internet Backbone, IN’None49.44.59.36192.168.0.1
55824NKN-CORE-NW NKN Core Network, IN’None49.44.59.36192.168.0.1
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3649.44.59.38125.22.47.125
55577BEAMTELE-AS-AP Atria Convergence Technologies pvt ltd, IN’49.44.59.3849.44.59.36192.168.1.254
134326AIRDESIGNBROADCAST-AS Airdesign Broadcast Media Pvt Ltd, IN’49.45.63.149.45.63.2208.67.222.222
138786CCBSPL-AS-IN Crystal Clear Broadband Services Pvt. Ltd., IN’49.45.63.149.45.63.28.8.8.8
56166IISERBNET-IN IISER Bhopal Campus, IN’49.44.59.3849.44.59.36172.30.1.2
24560AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services, IN’49.44.59.3849.44.59.36192.168.1.1
9829BSNL-NIB National Internet Backbone, IN’49.44.59.3649.44.59.38fdf6:a86d:4264::1
55836RELIANCEJIO-IN Reliance Jio Infocomm Limited, IN’49.44.59.3649.44.59.38192.168.31.1
139331DCORP-AS-AP DevelentCorp., IN’49.45.63.149.45.63.28.8.8.8
23860ALLIANCE-GATEWAY-AS-AP Alliance Broadband Services Pvt. Ltd., IN’49.44.59.3849.44.59.361.1.1.1
55836RELIANCEJIO-IN Reliance Jio Infocomm Limited, IN’49.45.63.149.45.63.28.8.4.4
17488HATHWAY-NET-AP Hathway IP Over Cable Internet, IN’49.44.59.3849.44.59.361.1.1.1
135718DISHAWAVESINFONET-AS DISHAWAVES INFONET PVT. LTD, IN’None49.44.59.361.1.1.1
4755TATACOMM-AS TATA Communications formerly VSNL is Leading ISP, IN’49.45.63.149.45.63.28.8.8.8
16509AMAZON-02, US’49.44.59.3849.44.59.36::1
15169GOOGLE, US’49.45.63.149.45.63.2::1
139331DCORP-AS-AP DevelentCorp., IN’49.44.59.3649.44.59.38::1
24309CABLELITE-AS-AP Atria Convergence Technologies Pvt. Ltd. Broadband Internet Service Provider INDIA, IN’49.45.63.249.45.63.1192.168.1.10
9498BBIL-AP BHARTI Airtel Ltd., IN’49.45.63.149.45.63.2192.168.139.245

internationalvowifi.jio.com also seems to indicate VoWiFi International calling, which resolves to 49.44.59.36 and 49.44.59.38 from my vantage point. The same resolves to 49.45.63.1 and 49.45.63.2 from every location that I’ve managed to check from outside India.

Looking at the results, most likely the issue is with how ns1.vowifi.jio.com & ns2.vowifi.jio.com are responding to  client subnet (EDNS0) in DNS queries.

If you enjoyed reading this blog post, you might find root hints vs RFC 8806 interesting.

DNS RPZ (Response Policy Zones) – Using DNS as a layer of defence – Part I

Update (06/08/2020)APNIC has published this post on their blog. Robbie Mitchell from APNIC was of great help in correcting a few things and polishing the article. You can read the Part 1 on the APNIC blog here

DNS(Domain Name System) is the crucial & ubiquitous fabric of the Internet.  While on the surface, users rely on accessing websites, apps, email etc underneath it’s the DNS database which provides the map for the Internet.

It’s fair to say that everything on the Internet begins with a DNS query. This means that the DNS is used for legitimate purposes and as well as abused by bad actors.

Adding a layer of security to a flat network

In the context of COVID-19, where most of us are working from home, security of the the devices & data being accessed from a hostile home network has become a major talking point over the last couple of months. The home network is atypical from an enterprise network from a security perspective and apart from its inherent flaws, it’s a flat network.

flat network is a computer network design approach that aims to reduce cost, maintenance and administration.[1] Flat networks are designed to reduce the number of routers and switches on a computer network by connecting the devices to a single switch instead of separate switches. Unlike a hierarchical network design, the network is not physically separated using different switches.
The topology of a flat network is not segmented or separated into different broadcast areas by using routers.

Wikipedia

Here is a representation of a flat network design,

The constraints of a flat network are,

  • No segmentation of traffic – Single broadcast domain
  • Easy & rapid propagation of malicious traffic within the network

One of the layers of security that can be brought into a flat network at an economical cost is by leveraging DNS. Before we look into how that can be implemented, here is a DNS primer for what happens when a domain name is accessed in a network,

Shift of the recursive resolvers

In the above diagrammatic representation, the part which is doing the most heavy lifting is the Recursive DNS Server or Recursive resolver. At the very beginning of the Internet, users themselves ran recursive resolvers on the machines or in the network. This model slowly shifted to the network operators (ISP’s) offering this as a bundled, free of cost offering along with the service. And the model has moved DNS resolution further away from the user with the advent of the Cloud/Quad DNS providers. To name a notable few, Google Public DNS (8.8.8.8, 8.8.4.4), Cloudflare (1.1.1.1, 1.0.0.1), Quad9(9.9.9.9) etc.

While each of these open resolvers services promote faster dns resolution, in reality they are still further away from the user from a round trip metric. Even though all of these open resolver services use IP Anycast, the proximity to the user cannot compete with a local resolver. In obvious terms, the recursive resolver which is in the users network or even the resolver provided by the Internet Service Provider will always be closest.

The one definitive advantage that the cloud/quad DNS open resolvers provide is the availability of a large cache.

If you aren’t convinced yet on running your own DNS resolver instead of outsourcing it to the cloud/quad DNS providers, I would urge you to read Why should I run my own DNS resolver?

And most importantly, if you want to leverage DNS Response Policy Zones (DNS Firewall) to add a layer of security in your network, you need to run a recursive resolver.

What is DNS Response Policy Zones(RPZ) ?

  • It’s currently an Internet-draft and not a standard yet. The latest draft is available here
  • It’s a vendor neutral – BIND, Unbound, PowerDNS Recursor support it
  • Allows policy to be applied to DNS queries. Set a differentiated route for the bad domains
  • Economical solution – a RaspberryPi can act as recursive resolver with DNS RPZ for the entire network – especially useful & low cost solution for home networks, SOHO etc

Just like the functioning of a firewall, RPZ is made up of TRIGGERS & ACTIONS.

This is all good but without threat intelligence data, a DNS Firewall doesn’t add any value.

Threat intelligence RPZ feeds

While there are many threat intelligence providers which provide a DNS RPZ feed, below are some of the free/community ones,

Update: Please refer to this blog post for an updated list of feeds.

Part II of this post will contain instructions for configuring a RPZ feed in ISC BIND9.

Chromium based browsers & DNS

While this is not something new, it perhaps has more significance because of the ever increasing market share of more than 60% of Chromium based browsers.

Chromium based browsers have a very uncanny method to check if the web browser is sitting behind a captive portal. And if you’re running a recursive resolver in your network with a large user base running Chromium based browsers (Google Chrome, Brave etc), it might even startle you if you observe the recursive resolver logs.

Here is a snippet from my unbound resolver as soon as I start Google Chrome on the machine(192.168.0.188),

Jun  3 11:16:31 root unbound: [1283:0] info: 192.168.0.188 pwpsfrn. A IN
Jun  3 11:16:31 root unbound: [1283:0] info: 192.168.0.188 yeytluindg. A IN
Jun  3 11:16:31 root unbound: [1283:0] info: 192.168.0.188 zkgtcrxrpfjcjxr. A IN

A research project at USC What’s In A Name? goes into some detail with the classification.

Here is the summary of the study,

Though the root server system handles this application-specific load sufficiently, it is clear that Chrome’s trick of using randomly generated names to discover whether it’s behind a captive portal contributes significantly to the traffic received at the root zone.

What’s in a name? – Wes Hardaker

33,384 open resolvers in India

The Shadowserver Foundation releases and updates a scan report containing results for open resolvers on the Internet. Open resolvers basically respond to DNS queries from anyone on the Internet. Open resolvers are bad for the Internet primarily because they are a catalyst in a DNS amplification attack.

A Domain Name Server (DNS) Amplification attack is a popular form of Distributed Denial of Service (DDoS), in which attackers use publicly accessible open DNS servers to flood a target system with DNS response traffic. The primary technique consists of an attacker sending a DNS name lookup request to an open DNS server with the source address spoofed to be the target’s address. When the DNS server sends the DNS record response, it is sent instead to the target.

Source

At the time of writing this, from an India perspective, there are 33,384 open resolvers. The number was 72,736 a couple of weeks ago.

Of the quantum, at that time,

ASNAS NameCount
AS9829BSNL-NIB National Internet Backbone77,736

So, what’s going on here ? Most likely, it’s a broken configuration in the CPE(Customer Premise Equipment) of AS9829 which is allowing DNS requests on the WAN IP address and performing recursion.

Most of the cheap Consumer Premise Equipment(CPE) devices that are bundled with the Internet connection run dnsmasq and the firmware never sees an update.

Interestingly, when I compare this with my own measurements, the number of IP addresses responding to port 53 in my results is much higher – 260,886. Though, I haven’t filtered the responses for IP addresses which are performing recursion. There could be IP addresses in the results which are configured as authoritative name servers and that’s perfectly valid.

For some reason, if you are running a DNS resolver on the Internet, strongly suggest that you restrict access by IP address/network.

A better approach is perhaps to configure the DNS resolver software on a RFC1918 private IP address & configure Wireguard/openvpn. Using this approach, the resolver is never exposed to the Internet while at the same time, devices can send DNS queries via the wireguard/openvpn tunnel.

If you found this blog post useful, you might find Shodan geoping and geodns – check ping & DNS resolution interesting.