Net.HttpRequest - blank response?
Hi,
I am trying to create a script to connect to a website and extract all the content of the page. And in future to connect to an API.
DO httprequest.Get("http://www.intersystems.com",2)
Write ">"_httprequest.HttpResponse
But the output is a wonderful blank HttpResponse
Any ideas?
Thanks
There are some mistakes.
#1 the links should be "HTTPS://www.intersystems.com" and you didn't set a ssl/tls config.
if you use
Do $system.OBJ.DisplayError(sc)
you get ERROR #6159: ===> SSL missing
#2 HttpResponse is an ObjectReferce not a Property
ZW res
res=<OBJECT REFERENCE>[3@%Net.HttpResponse]
+----------------- general information ---------------
| oref value: 3
| class name: %Net.HttpResponse
| reference count: 3
+----------------- attribute values ------------------
| ContentBoundary = ""
| ContentInfo = ""
| ContentLength = 178
| ContentType = "text/html"
| Data = "4@%Stream.GlobalCharacter" ;;; here is your reply
|Headers("CONNECTION") = "keep-alive"
|Headers("CONTENT-LENGTH") = 178
|Headers("CONTENT-TYPE") = "text/html"
| Headers("DATE") = "Wed, 01 Aug 2018 15:25:05 GMT"
|Headers("LOCATION") = "https://www.intersystems.com/"
| Headers("SERVER") = "nginx"
| Headers("X-TYPE") = "default"
| HttpVersion = "HTTP/1.1"
| ReasonPhrase = "Moved Permanently"
| StatusCode = 301
| StatusLine = "HTTP/1.1 301 Moved Permanently"
+-----------------------------------------------------
The content is in a Stream!!
So Write is totally inappropriate to show it. Instead:
HTTP/1.1 301 Moved Permanently
CONNECTION: keep-alive
CONTENT-LENGTH: 178
CONTENT-TYPE: text/html
DATE: Wed, 01 Aug 2018 15:25:05 GMT
LOCATION: https://www.intersystems.com/
SERVER:nginx
X-TYPE: default
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
Yo are absolutely right although I am still confused.
I copied httprequest.Get("http://www.intersystems.com",2) from one example found in the documentation so I was expected to work. Also, to get data from a website don't think you have to use https or SSL right? I tried with different APIs in other applications and just passing the right URL I was able to retreive the data.
About httprequest.HttpResponse you are right again... I was using httprequest.HttpResponse.Data and outputtodevice but I was having an error as the object did not exist...so I tried to check if the object was really created or not...
What would be the right script to connect then? I am trying to get data from this API https://api.postcodes.io/postcodes/
Thanks
Thanks for your help... I think the problem I am having is not with my code but with ensemble or IT. Apparently somehow the connection to external content is blocked. I can access to the websites with no issues from web browsers but when trying to access from studio or test SSL from ensemble...error!!
Is there any security configuration to block that kind of access in ensemble?
Thanks
I have tried to created an SSL/TLS configuration but not sure why is not working:
It does not matter what I try to test it...always returns the same error:
ERROR #989: SSL connection failed, make sure server address and port (not url) is specified
I tried "intersystems.com" 443, "postcodes.io" 443...nothing
Those sites I am trying to connect to....don't really need anything special to connect...just url
Thanks
You have to test on server = "www.intersystem.com" not just domain.
443 is ok
USER>s req=##class(%Net.HttpRequest).%New()
USER>s req.SSLConfiguration="SSL"
USER>set sc=req.Get("https://www.intersystems.com")
USER>zw sc
sc=1
USER>set data=req.HttpResponse.Data
USER>do data.OutpuToDevice()
Thanks for the help:
1. I have tried from the server where ensemble is running. In IE, I can access www.intersystems.com but when trying from studio...nothing blank response...all from the same matchine!
2. I tried that and yes... ERROR #6059: Unable to open TCP/IP socket to server www.intersystems.com:80 so I think you are right but I cannot believe the problem is no visibility when in IE I can access the website with no issues.
I have tried by IP, apparently 38.111.0.52
so in IE: http://38.111.0.52:80 => ok, https://38.111.0.52:443 => ok
but using:
Set httprequest.Server = "38.111.0.52"
Set httprequest.Port=80
set status = httprequest.Get("/",2)
DO $system.OBJ.DisplayError(status)
I have no idea what's going on
Hi,
So I checked that the web site will receive a ping, some don't for security purposes, and it seems ok:
ping www.intersystems.com
PING e9933.g.akamaiedge.net (104.103.202.60): 56 data bytes
64 bytes from 104.103.202.60: icmp_seq=0 ttl=56 time=80.280 ms
64 bytes from 104.103.202.60: icmp_seq=1 ttl=56 time=76.565 ms
64 bytes from 104.103.202.60: icmp_seq=2 ttl=56 time=53.358 ms
ping 104.103.202.60
PING 104.103.202.60 (104.103.202.60): 56 data bytes
64 bytes from 104.103.202.60: icmp_seq=0 ttl=56 time=17.981 ms
64 bytes from 104.103.202.60: icmp_seq=1 ttl=56 time=17.399 ms
64 bytes from 104.103.202.60: icmp_seq=2 ttl=56 time=86.598 ms
64 bytes from 104.103.202.60: icmp_seq=3 ttl=56 time=53.091 ms
64 bytes from 104.103.202.60: icmp_seq=4 ttl=56 time=54.445 ms
Try opening up a terminal on the Ensemble box and PING www.intersystems.com. If that works then we know that the network is ok.
Good Luck
Iain
not working but ping could be blocked in a firewall...and when I tried from my local PC, ping did not work either...
I could be wrong but I am pretty sure is something blocking that connection but maybe based on the protocol...not sure...what I cannot understand is why browse to the web site works but httprequest not.
Thanks
It's possible that a proxy is in use for web access. Browsers can generally auto-discover and configure for those, but other web clients can't ... at least most of the time.
Apologies regarding the DNS name. I've just remembered that you're trying to get a response from 'www.intersystems.com' so the DNS wouldn't have an effect. What you should check is that the Ensemble instance has access to the internet.
There isn't anything in Ensemble that will prevent access, it might be something at the O/S or network level.
Iain
Ok, so a couple of things to check:
1. When you say that you can connect from a browser but not from Ensemble, is the browser that you're using on the same computer as Ensemble?
2. Instead of a 'do httprequest.Get(...)' use 'set status=httprequest.Get(...)' and then inspect the value of 'status'. You can do 'do $system.OBJ.DisplayError(status)' in order to get a readable output. If the result is that you cannot open a TCP/IP socket then Ensemble cannot see the other computer. Otherwise the status should give you more information.
Are you addressing the other computer by its IP address or its DNS name? If you're using the DNS name then try using the IP address. If that works then edit your 'hosts' file to put the name and IP address in there and go back to using the DNS name to get to the other computer.
Let me know how you get on.
Iain
This will work. The second parameter of '2' indicates test mode and will write out the response to the current device
set httprequest.Server = "www.intersystems.com"
set status = httprequest.Get("/",2)
You should see something like this:
HTTP/1.1 301 Moved Permanently
CONNECTION: keep-alive
CONTENT-LENGTH: 178
CONTENT-TYPE: text/html
DATE: Thu, 02 Aug 2018 10:21:56 GMT
LOCATION: https://www.intersystems.com/
SERVER: nginx
X-TYPE: default
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
Alternatively you can do this:
Setup an SSL/TLS Configuration. This is very simple and doesn't have any values apart from a name. Mine is called 'test'
Then use the following code:
set httprequest.Server = "www.intersystems.com"
set httprequest.SSLConfiguration = "test"
set httprequest.Https = 1
set status = httprequest.Get("/")
if status {
do httprequest.HttpResponse.Data.Rewind()
do {
write httprequest.HttpResponse.Data.Read()
} while 'httprequest.HttpResponse.Data.AtEnd
}
Social networks
InterSystems resources
Log in or sign up
Log in or create a new account to continue
Log in or sign up
Log in or create a new account to continue
Log in or sign up
Log in or create a new account to continue