Hi,
What to do if the domain name of one of my webserver, that me and some lab members use for work related stuff, is no longer resolved by our university DNS? When I first noticed it, I could see no resolution at all while now the domain resolves to a wrong IP. The site can be normally reached on any other network so there is no problem on my side I think.
Should I just wait (now more than 24 hours) or should I try anything? I am entitled to complain to our IT even though the issue is only with this not-really-professional FreeDNS subdomain?
EDIT: apparently some automatism marked this domain as malicious (absolutely it is not, not willingly and not compromised) and somehow DNS resolves to CNAME sinkhole.paloaltonetworks.com.
You must log in or register to comment.
Sounds like your university is using a Palo Alto Next Gen Firewall which is intercepting DNS requests and responding with the sinkhole FQDN for anything they deem malicious or suspicious. You can try to override this with DNS over HTTPS but they may also be blocking that. Standard security stuff. You can also probably try to open an IT ticket and request that they whitelist the domain.
So it seems. Do you think this was from the detected user activity? A colleague reported it was using it and it stopped working from one second to the next. Maybe some of his traffic looked suspicious? I am opening a ticket in any case today.
That is possible as well. Those firewalls are capable of packet inspection. If you are using personal devices it won’t be able to see much if you are using encryption in transit but if you are using University provided machines there is a good chance they can inspect all the data you are sending and receiving.
This may come down to details of their policies and how they interact and support each department. If it’s for your official work, and I’d say start with a ticket and if they resist then push it up the flag pole and don’t stop. (Assuming you’re not one,) Your PI ought to fight like hell to make sure their employees can do their jobs, and the chair fight to make sure their researchers can run their labs, and the dean much the same, but throwing heavier punches each step up. Really shouldn’t get to that point, but if you can’t do your job, rattle the cages until you can.
I already had contacts with our IT. I originally asked if they could host this service for us as it seemed the normal thing to do. They do not support anything custom (i.e. anything which is not a wordpress site) and just to give me a fourth level subdomain they wanted signatures from half the administration above me. That’s why I’m rogue with selfhosting also work stuff. But I think I can still complain just because their DNS gives back random IPs. This could even be hijacking, no?
I would probably send along the output of
dig your.domain @uni-ipanddig your.domain @8.8.8.8anddig your.domain @1.1.1.1anddig your.domain @your-domains-authoritative-dns-serverif you have that or some similar DNS client tool installed that allows direct requests to specific servers. If you don’t know the authoritative DNS servers for your domain, those are the ones in the NS records.Nice, I am routed to sinkhole.paloaltonetworks.com I am a malicious domain apparently.
Are you hosting a service that is not under your organizations official domain or something? It is common security practice to block newly created domains which may be why your domain is blacklisted if you only recently stood it up.
I think there are many levels to approach this problem. First off the obvious investigate why your org DNS is having issues. This is IT request they should fix that. They should have SLA on this critical service and not fixing it should escalate to management. There may be many reasons why resolver is not working specially in complex multi site setups. This is the best option as it solves this and probably other DNS related issues.
The rogue approach: On other side if you only host service for handful of users that you personally know and you have ability to edit your hosts file, you can bypass DNS completely. This isn’t ideal as it has to be done one every system and in case your IP changes you will have to do it again. It would largely depend on your level of access to system. If you even can change hosts file.
Alternative crazy idea is to host your own DNS. Change DNS setting on your network configuration. Then point your dns to your org dns. Same problem as hosts file you will need to do that for all systems that need connectivity.
Expanding on own DNS approach you could go as far as hosting your own network. WiFi or switch in case you need Ethernet cable connection. You can buy used enterprise equipment for cheap plug it in l, configure to point to your own DNS and anyone connected to your network would have your settings. Of course this is super shadow IT and I would discourage from pursuing that.
Less crazy and rogue option is to use something like tailscale (or similar) which would have DNS (magic dns). You would need agent installed on every client.
Thanks for the detailed answer, a lot of suggestions are great but unfortunately a bit unpractical. Changing etc/hosts is at the moment the only think working and if the issue is not fixed soon I will suggest to the users that are willing to do so. I would not go as far as asking people to install VPNs and I am pretty sure that buildin a rogue wifi/LAN network will be against any corporate policy and I will be fired :D
Why are you using a crappy uni DNS? Why not 1.1.1.1 or OpenDNS or even Google’s 8.8.8.8?
That doesn’t help when you run the server and other people have to access your website.
Well, the main point is I would need to manually change this for tens of pcs and its not my job, moreover other people should to the same on theirs. Nevertheless, I just tried 8.8.8.8 on a couple of PCs and I have the same issue! It appears that my DNS setting is irrelevant as it is overwritten down the chain, the only way I can reach the site is put the line in etc/hosts. Could it be?
Yes, your uni might intercept communication on port 53 and reroute it to their DNS servers. It’s possible.
Do you have a static IP? If not, have you tried some kind of dynamic DNS like DuckDNS?
The IP is static, and is resolved properly everywhere outside my university network
An the issue is only inside the network? I’d complain to IT about that, yeah. Maybe they are overriding the DNS record with their own DNS server or something.
Can you set your own DNS servers on your client devices? Does cloudflare or quad9 resolve it?
I think this is exactly the case, they have some issues with the DNS server and, as some other comments indicate it is possible, they reset my settings for DNS servers at router level. So nor cloudflare or others can help, only the line in etc/hosts works