How would I capture the results of a web request without using a browser

[Martin Francom]

How would I capture the results of a web request without using a browser
Is there a way I can capture the results of a web request without using a browser. I need to make a web call such as:

-https:// webservices.lexi.com/rest/ref/interact/analyze?generic=2093,1051,1233

what is returned is an XML file that I want to parse using my PowerBasic program.

Is there a way to do this without a browser?

[Liew Laurence]

Hi,

Below is a simple CGI test script I wrote for testing my CGI.exe.

Cheers!
L

Code Select Expand



#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

    LOCAL hSocket AS LONG
    LOCAL dsp_server, sBuffer, sTemp, headerData, bodyData AS STRING

    dsp_server = "localhost"

    'TCP OPEN PORT %DSP_PORT AT $DSP_SERVER AS #hSocket TIMEOUT %DSP_TIMEOUT_CLI
    TCP OPEN PORT 80 AT dsp_server AS #hSocket TIMEOUT 3600
    IF ERR THEN
        MSGBOX "Error opening " + dsp_server + " " + STR$(ERR)
        EXIT FUNCTION
    END IF

    sBuffer = ""
    sBuffer = "code=1000"

    TCP PRINT #hSocket, "POST " + "/DSPComputeService2.exe" + " HTTP/1.1"
    TCP PRINT #hSocket, "Accept: */*"
    TCP PRINT #hSocket, "Accept-Language: en-us"
    TCP PRINT #hSocket, "Host: " + dsp_server
    TCP PRINT #hSocket, "Referer: http://localhost/"
    TCP PRINT #hSocket, "UserAgent:CGI TEST"
    TCP PRINT #hSocket, "Content-Type: application/x-www-form-urlencoded"
    TCP PRINT #hSocket, "Content-Length: " + FORMAT$(LEN(sBuffer))
    TCP PRINT #hSocket, ""
    TCP PRINT #hSocket, sBuffer


    IF ERR THEN
        MSGBOX "ERROR: Sending data to CGI. Error number: " & STR$(ERR)
    ELSE
        MSGBOX "CGI POST command OKAY"
    END IF

    MSGBOX "CGI data sent: " & sBuffer

    sBuffer = ""
    DO
        TCP RECV #hSocket, 4096, sBuffer
        sTemp = sTemp + sBuffer
    LOOP WHILE LEN(sBuffer)

    TCP CLOSE #hSocket

    IF ERR THEN
        MSGBOX "ERROR receiving data from DSP server. Error number: " & STR$(ERR)
        EXIT FUNCTION
    END IF

    headerData  = EXTRACT$(sTemp, $CRLF + $CRLF)
    bodyData    = REMAIN$(sTemp, $CRLF + $CRLF)

    'DEBUG
    MSGBOX "Header: " & headerData
    MSGBOX "Body: "  & bodyData

END FUNCTION   




[Michael Stefanik]

In this specific case, one of the issues is that the connection is secure (using TLS) so you can't just use the native socket functions. It also looks like access to the site requires authentication. Here's how it could be done using SocketTools:

Code Select Expand


#Compile Exe
#Dim All
#Include "cstools7.inc"

Function PBMain () As Long
    Dim hClient As Dword
    Dim szUrl As StringZ * 128
    Dim szBuffer As StringZ * 4096
    Dim nResult As Long

    szUrl = "https://webservices.lexi.com/rest/ref/interact/analyze?generic=2093,1051,1233"

    ' Initialize the library
    If IsFalse(HttpInitialize("", %NULL)) Then
        MsgBox("Unable to initialize SocketTools")
        Exit Function
    End If

    ' Connect to the site using the specified URL
    hClient = HttpConnectUrl(szUrl, %HTTP_TIMEOUT, %HTTP_OPTION_DEFAULT)
    If hClient = %INVALID_CLIENT Then
        MsgBox("Unable to connect to server")
        Exit Function
    End If

    ' Access to the site requires authentication with a username and password
    HttpAuthenticate(hClient, %HTTP_AUTH_BASIC, "myname", "secret")

    ' Return a copy of the XML data in the szBuffer variable
    nResult = HttpGetText(hClient, szUrl, szBuffer, 4096)
    If nResult = %HTTP_ERROR Then
        MsgBox("Unable to retrieve document")
        HttpDisconnect(hClient)
        Exit Function
    End If

    ' Disconnect from the server
    HttpDisconnect(hClient)
    MsgBox(szBuffer)
End Function


Microsoft's WinInet API could also be used, I think the PowerBasic forums have some examples floating around there.