使用 Winsock2 向具有域 HTTPS 的网站发送 GET 请求

Question

我正在尝试向 Web 服务器发出 GET 请求，我已经成功地处理了具有 HTTP 域的网站。剩下的问题是我无法成功地向具有 HTTPS 域的网站发送请求。

我必须在 C++ 中使用 Winsock 来完成这项任务。这是我已经成功实现 HTTP 的代码：

void GET_request(URL url)
{
    WSADATA wsaData;
    SOCKET Socket;
    SOCKADDR_IN SockAddr;
    struct hostent* localHost;
    string request_header;
    char* localIP;
    int find_index;
    url.URL_detach();

    //Create a Request Header.
    request_header = "GET "+ url.path + " HTTP/1.0\r\n";
    request_header += "Host: " + url.host + "\r\n";
    request_header += "Connection: close\r\n";
    request_header += "Cache-Control: max-age=0\r\n";
    request_header += "Upgrade-Insecure-Requests: 1\r\n";
    request_header += "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36\r\n";
    request_header += "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\n";
    request_header += "Accept-Encoding: gzip, deflate\r\n";
    request_header += "\r\n";

    if (WSAStartup(MAKEWORD(2, 2), &wsaData) != 0) {
        cout << "WSAStartup failed.\n";
        system("pause");
    }

    Socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);         

    localHost = gethostbyname(url.host.c_str());                
    localIP = inet_ntoa(*(struct in_addr *)*localHost->h_addr_list);        //Return IP of HOST
    SockAddr.sin_port = htons(80);                                          //PORT 80 (HTTP)
    SockAddr.sin_family = AF_INET;                                          // TCP/IP
    SockAddr.sin_addr.s_addr = inet_addr(localIP);                          

    if (connect(Socket, (SOCKADDR*)(&SockAddr), sizeof(SockAddr)) != 0)     
    {
        cout << "Could not connect to Server";
        system("pause");
    }

    send(Socket, request_header.c_str(), strlen(request_header.c_str()), 0);    


    int nDataLength;
    while ((nDataLength = recv(Socket, buffer, 10000, 0)) > 0) {
        int i = 0;
        while (buffer[i] >= 32 || buffer[i] == '\n' || buffer[i] == '\r') {
            Response_Header += buffer[i];
            i += 1;
        }
        find_index = Response_Header.find("</html>");
    }
    Response_Header.erase(find_index + 7);
    closesocket(Socket);
    WSACleanup();

}

Answer 1

感谢hugtech 提出的问题、代码以及使用 HTTP 协议的工作。也非常感谢Remy Lebeau 告知我们有关 SSL 和 TLS 握手的信息。我的研究从他们那里起飞。

在我的例子中，发送原始 HTTP GET 命令导致 WinSock send(...) 以零失败。想象一下，我什至不允许发送那些未加密的东西。使用从站点复制的 ClientHello 后，我得到 www.youtube.com 给我发回一个 ServerHello。我不断地调整、编译我的代码并进行测试；我希望 Youtube 不会感到沮丧并要求提供验证码。

然后我就能够成功地修改 ClientHello。我最后简单地取出了扩展，因为它们是可选的，而且我不会支持它们。并修改了长度变量以适应调整。

// Client Hello I copied from the site
//uint8_t clientHello[] = { 0x16, 0x03, 0x01, 0x00, 0xa5, 0x01, 0x00, 0x00, 0xa1, 0x03, 0x03, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x00, 0x00, 0x20, 0xcc, 0xa8, 0xcc, 0xa9, 0xc0, 0x2f, 0xc0, 0x30, 0xc0, 0x2b, 0xc0, 0x2c, 0xc0, 0x13, 0xc0, 0x09, 0xc0, 0x14, 0xc0, 0x0a, 0x00, 0x9c, 0x00, 0x9d, 0x00, 0x2f, 0x00, 0x35, 0xc0, 0x12, 0x00, 0x0a, 0x01, 0x00, 0x00, 0x58, 0x00, 0x00, 0x00, 0x18, 0x00, 0x16, 0x00, 0x00, 0x13, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, 0x75, 0x6c, 0x66, 0x68, 0x65, 0x69, 0x6d, 0x2e, 0x6e, 0x65, 0x74, 0x00, 0x05, 0x00, 0x05, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x0a, 0x00, 0x08, 0x00, 0x1d, 0x00, 0x17, 0x00, 0x18, 0x00, 0x19, 0x00, 0x0b, 0x00, 0x02, 0x01, 0x00, 0x00, 0x0d, 0x00, 0x12, 0x00, 0x10, 0x04, 0x01, 0x04, 0x03, 0x05, 0x01, 0x05, 0x03, 0x06, 0x01, 0x06, 0x03, 0x02, 0x01, 0x02, 0x03, 0xff, 0x01, 0x00, 0x01, 0x00, 0x00, 0x12, 0x00, 0x00, };

// modified client hello
// unnecessary extension code has been erased,
// just as how stack overflow moderators will erase
// my unnecessary wordings and sentencing.
uint8_t clientHello[] =
{
 //Record Header
    0x16, //type: handshake
    0x03, 0x01, //TLS version:
    0x00, 75, //HandShake Length

 //Handshake Header
    0x01, //type: client hello
    0x00, 0x00, 71, //handshake data len

 // Client Version
    0x03, 0x03, // TLS 1.2 //actual TLS version client uses

 // Client Random Garbage 
 // Here I see 32 bytes being used; but I don't know if 32 bytes is mandatory.
 // But if it is not mandatory how would one know when the random garbage ends? 
    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,

 //Session ID
 // This ID is used to logon back from a previous session.
 // This eliminates certain computations as per what I read.
 // Here we use zero to say this is a fresh session with the server.
    0x00,

 // Cypher Suites
 // A list of Cypher Suite used to do the encryption. 
 // Cypher Suites are represented by two bytes.
 // Example 0xCCA8 is TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 Cypher Suite.
    0x00, 0x20, // count of bytes
    0xcc, 0xa8, 0xcc, 0xa9, 0xc0, 0x2f, 0xc0, 0x30, 0xc0, 0x2b, 0xc0, 0x2c, 0xc0, 0x13, 0xc0, 0x09,
    0xc0, 0x14, 0xc0, 0x0a, 0x00, 0x9c, 0x00, 0x9d, 0x00, 0x2f, 0x00, 0x35, 0xc0, 0x12, 0x00, 0x0a,

 // Compression Methods
 // A list of compression methods.
 // Here we have only one compression method which is zero for no compression.
    0x01, //bytes of compression methods
    0x00,

// After this we have one long list of Extension information which
// I have no interest in at all.
};

size_t clientHelloLen = sizeof(clientHello);

待续...
服务器你好

我已经阅读了 Joshua Davis 的 350 页的书，实现 SSL_TLS，结果发现 TLS 握手比任何人都解释的要复杂得多。我设法为www.youtube.com 准备了我自己的ClientHello 消息并取回了一个有效的ServerHello 和ServerCertificate(s)。但是当我用www.example.org 和www.apachefriends.org 尝试这个时，我没有收到任何消息。然后，我尝试了从https://tls.ulfheim.net/ 复制的原始 ClientHello，并取回了格式正确的 ServerHello 和 ServerCertificate，我可以使用我的自定义例程进行打印。问题是从服务器获得响应需要大约一分钟；我注意到证书看起来很可疑。它由同一个实体发行和发行。它还指出这是一个假证书控制。

然后我使用 OpenSSL 运行以下命令。 openssl s_client -connect "www.apachefriends.org:https"。我立即收到了服务器的回复！这意味着该站点正在拒绝我的自定义 ClientHello 消息。这意味着我浪费了我的周末来学习 Joshua Davis 的书，而应该阅读一本展示如何使用 OpenSSL 的书。

更新：
我终于可以开始工作了，但是 HTTPS、TLS/SSL 握手实在是太复杂了。我很生气，我研究了 ClientHello、ServerHello、ServerCertificate 的详细信息，但根本没有用。所有这些都被 OpenSSL 抽象出来了。并且出于某种原因，网站拒绝接受我的自定义 ClientHello，即使我完全按照 Joshua Davis 展示的那样做。我也对 OpenSSL 抽象出 Linux 和 Win32 SOCKETS 感到难过；所以学习这些细节只是为了学术。

以下是我使用 OpenSSL 从服务器获取文件的代码。请注意，在下载文件时，服务器通常会将您重定向到实际拥有该文件的另一台服务器。在这里，我对重定向进行了硬编码。

#include <Jav/test.h> // rep(...)

// My custom library for handling files
// You can substitute this with std::fstream, FILE*, or
// any other file library
#include <Jav/file/file.h>

// Custom exception class, Jav::Error
// derived from std::exception
#include <Jav/error/error.h>

//Trying to get _fileno to be defined
// as it is needed by openssl/applink.c
#undef __STRICT_ANSI__ 

// Undefing strict ansi failed so I simply
// declared _fileno
extern "C"{
_CRTIMP int __cdecl __MINGW_NOTHROW _fileno (FILE*);
}

# include  <openssl/bio.h>
# include  <openssl/ssl.h>
# include  <openssl/err.h>

// Yep we actually include a source file. 
// Other wise openssl gives "no app link" error during runtime.
// I tried adding applink.c to src instead of including it
// but got linker errors.
# include  <openssl/applink.c>

void init_openssl_library()
{
 SSL_library_init();
 SSL_load_error_strings();
 OPENSSL_config(NULL);
}

void on_openssl_fail(int line)
{
 ERR_print_errors_fp(stderr);
 throw Jav::Error("problem at line %i\n",line);
}

#define HOST_REDIRECTED "redirectedexamle.org" 
#define RESOURCE        "/exampleresource.txt"

int main()
{
    SSL_CTX *ctx = NULL;
    BIO *web = NULL, *out = NULL;
    SSL *ssl = NULL;

    init_openssl_library();

    const SSL_METHOD *method = SSLv23_method();
    if(method == NULL) on_openssl_fail(__LINE__);

    ctx = SSL_CTX_new(method);
    if(ctx == NULL) on_openssl_fail(__LINE__);

    // Cannot fail???
    //SSL_CTX_set_verify(ctx,SSL_VERIFY_PEER,verify_callback);
    //SSL_CTX_set_verify_depth(ctx,4);

    const long flags = SSL_OP_NO_SSLv2|SSL_OP_NO_SSLv3|SSL_OP_NO_COMPRESSION;
    SSL_CTX_set_options(ctx,flags);

    web = BIO_new_ssl_connect(ctx);
    if(web == NULL) on_openssl_fail(__LINE__);

    if(BIO_set_conn_hostname(web,HOST":https") != 1)
        on_openssl_fail(__LINE__);

    BIO_get_ssl(web,&ssl);
    if(ssl == NULL) on_openssl_fail(__LINE__);

    const char *PREFERRED_CIPHERS = "HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4";
    if(SSL_set_cipher_list(ssl,PREFERRED_CIPHERS) != 1)
        on_openssl_fail(__LINE__);

    if(SSL_set_tlsext_host_name(ssl,HOST) != 1)
        on_openssl_fail(__LINE__);

    //out = BIO_new_fp(stdout,BIO_NOCLOSE);
    //if(out == NULL) on_openssl_fail(__LINE__);

    if(BIO_do_connect(web) != 1) on_openssl_fail(__LINE__);
    if(BIO_do_handshake(web) != 1) on_openssl_fail(__LINE__);

    X509 *cert = SSL_get_peer_certificate(ssl);
    if(cert == NULL) on_openssl_fail(__LINE__);
    X509_free(cert);

    //if(SSL_get_verify_result(ssl) != X509_V_OK) on_openssl_fail(__LINE__);

    int len = 10000;
    Jav::cstring buf(len);

    auto cmd = "GET "RESOURCE" HTTP/1.1\r\n"
               "Host: "HOST"\r\n"
               "Connection: close\r\n\r\n";
      
    if(BIO_puts(web, cmd) <= 0)
    {
        rep("bio write failed");

        if(! BIO_should_retry(web))
        {
          rep("bio should retry");
        }
        else return 1;
    }

    int total = 0;
    Jav::wFile file = Jav::createNewFile<Jav::wFile>("download.txt");

    while(true)
    {
        int numRead = BIO_read(web, buf, len);
        total += numRead;

        if(numRead == 0)
        {
         rep("No more data read");
         if(total) break;
         else return 1;
        }
        else if(numRead < 0)
        {
            rep("bio read failed");

            if(! BIO_should_retry(web))
            {
              rep("bio should retry");
        int numRead = BIO_read(out, buf, len);
        rep("num read",numRead);
            on_openssl_fail(__LINE__);
            }
            else return 1;
        }
        else rep("num read",numRead);

        file.write(buf,numRead);
    }

//Jav::wFile file = Jav::createFile<Jav::wFile>("output.txt");
//file.write(buf,numRead);
    BIO_free(out);
    BIO_free(web);
    SSL_CTX_free(ctx);
}

Answer 2

Jav HTTPS 网络：解决方案

我学习 HTTPS 的全部目的是创建一个下载器，以便在我的自定义安装程序中使用。我要做的就是展示我的代码；其中包括我的自定义库。您可以轻松地将我的自定义库代码替换为您自己的。

为了完成这项工作，我将使用 OpenSSL SDK。该库抽象出 Linux/Win32 套接字以及 TLS 握手和证书验证。这意味着您无需学习 Brian Hall - Beej 的网络编程或 Anthony Jones - Windows 网络编程。而且您绝对不需要学习 Joshua Jones - 实施 SSL_TLS。

您只需要学习关于 HTTP 的 TutorialPoints.com 教程。我从来没有读过所有的 TutorialPoints.com 教程。我简单的看一下关于 GET 命令和 HTTP Server Resonse 的五种类型的部分。有时需要 POST 命令来下载文件，但在我的情况下不需要。

• 100:STATUS_PROCESSING
• 200:STATUS_OK
• 300:STATUS_REDIRECTED
• 400:STATUS_FAILED
• 500:STATUS_SERVER_ERROR

在下载文件的情况下，必然会得到STATUS_REDIRECTED；似乎服务器让其他专用服务器负责存储要下载的大文件。

最后说明：此代码使用 GNU、Mingw 扩展。即能够将 goto 标签的地址复制到 void 指针中；这简化了我的条件 goto 分支。您可以改用标志。您甚至可以消除 goto 并使用自己的机制。我使用了 goto，因为它使代码更容易编写。

Jav/file/buffer.h

#ifndef JAV_BUFFER_HPP
#define JAV_BUFFER_HPP

#include <Jav/bits.h> /* bitMove */

namespace Jav {

enum { KB=1024, MB=KB*1024, GB=MB*1024 };

/** Represents a chunk of data from a particular source. Eg: file, network
  *
  * Buffer is not meant to operate as dynamic memory,
  * that allocates and reallocates memory to constantly increase/ decrease capacity.
  * Consider using std::string for such purpose.
*/
struct Buffer
{
 uint8_t *begin; // begin of allocated memory
 uint8_t *pos;   // position in buffer
 uint8_t *end;   // end of data written to buffer
 uint8_t *max;   // end of allocated memory

 Buffer();
 explicit Buffer(std::size_t size);
 Buffer(std::size_t size,std::size_t max);
 Buffer(Buffer &&move);
 Buffer& operator= (Buffer &&move);
 ~Buffer();

 explicit operator bool()const { return begin; }
 bool isEmpty()const           { return begin == end; }
 std::size_t size()const       { return end - begin; }
 std::size_t unread()const     { return pos >= end ? 0 : end - pos; }
 std::size_t space()const      { return max - pos; }
 std::size_t freeSpace()const  { return max - end; }
 std::size_t capacity()const   { return max - begin; }
 void clear()                   { end = pos = begin; }
};


/** Represents a chunk of data from a particular source. Eg: file, network */
class FileBuffer
{
 public:
    FileBuffer();
    explicit FileBuffer(std::size_t size);
    FileBuffer(std::size_t size,std::size_t max);
    FileBuffer(FileBuffer &&move);
    FileBuffer& operator= (FileBuffer &&move);
    ~FileBuffer();

 public:
    explicit operator bool()const { return begin; }
    std::size_t size()const       { return end - begin; }
    std::size_t unread()const     { return pos >= end ? 0 : end - pos; }
    std::size_t space()const      { return max - pos; }
    std::size_t freeSpace()const  { return max - end; }
    std::size_t capacity()const   { return max - begin; }
    void clear()                    { end = pos = begin; }

 private:
    int64_t fpos;   // pos in file
    uint8_t *begin; // begin of allocated memory
    uint8_t *pos;   // position in buffer
    uint8_t *end;   // end of data written to buffer
    uint8_t *max;   // end of allocated memory

 private:
    friend class FileBase;
    friend class ReadableFile;
    friend class WritableFile;
};


/// Read From Buffer

void* read   (Buffer&, void *dest, size_t dest_size);
void* readBeg(Buffer&, void *dest, size_t dest_size);
void* read   (Buffer&, int offset, void *dest, size_t dest_size);
void* readBeg(Buffer&, int offset, void *dest, size_t dest_size);


/// Write Into Buffer

void write(Buffer&,const void *src,size_t);
void writeBeg(Buffer&,const void *src,size_t);
void writeEnd(Buffer&,const void *src,size_t);
void insert(Buffer&,const void *src,size_t);
void insertBeg(Buffer&,const void *src,size_t);

void write(Buffer&,int offset,const void *src,size_t);
void writeBeg(Buffer&,int offset,const void *src,size_t);
void writeEnd(Buffer&,int offset,const void *src,size_t);
void insert(Buffer&,int offset,const void *src,size_t);
void insertBeg(Buffer&,int offset,const void *src,size_t);


/// Data Management
void notifyDataWritten(Buffer&,std::size_t);
void notifyDataRead(Buffer&,std::size_t);
bool shiftBytes(Buffer&,byte *from,byte *to);


/// Size Management
void setCapacity(Buffer&,std::size_t);


#include "Buffer.inl"

}

#endif // JAV_BUFFER_HPP

Jav/string/parser.h

#ifndef JAV_PARSER_HPP
#define JAV_PARSER_HPP

#include <Jav/string/cstring.h>
#include <Jav/error/error.h>
#include <string>
#include <stdio.h>

namespace Jav {

class Parser
{
 public:
    Parser(const char *bytes) : bytes(bytes) {}
    Parser(const Parser &parser) : bytes(parser.bytes) {}

 public:
    const char* skipSpaces(const char *it);
    const char* skipAllSpaces(const char *it);
    const char* nextLine(const char *it);
    ...

 public:
    const char* skipSpaces(const char *it, const char *end);
    const char* skipAllSpaces(const char *it, const char *end);
    const char* nextLine(const char *it, const char *end);
    ...

 protected:
    const char *bytes;
};
...

}

#endif // JAV_PARSER_HPP

Net.h

#ifndef JAV_APP_NET_HPP
#define JAV_APP_NET_HPP

#include <Jav/string/parser.h>
#include <Jav/string/cstring.h>
#include <Jav/file/buffer.h>
#include <openssl/bio.h>
#include <openssl/ssl.h>


enum { STATUS_UNKNOWN, STATUS_OK, STATUS_REDIRECT, STATUS_FAIL  };

struct Url
{
 Jav::cstring port;
 Jav::cstring host;
 Jav::cstring path;
};

bool parseUrl(Url&,const char *url);

struct Net
{
 public:
    Net(const char *url);
    Net(Net&&);
    Net& operator=(Net&&);
    ~Net();

 public:
    void requestFile();
    size_t recieveResponse(Jav::Buffer&);

 private:
    void on_openssl_fail(int line);

 private:
    Url url;
    SSL_CTX *ctx;
    BIO *web;
    SSL *ssl;
};

struct ServerResponseParser : Jav::Parser
{
    int status;
    size_t content_size;
    Jav::cstring content_url;
    Jav::cstring status_string;

    ServerResponseParser() : Jav::Parser(NULL) {}
    bool parse(Jav::Buffer &response,Net &store);

 protected:
    bool parseStatusLine(Jav::Buffer &response,Net &store);
    bool parseAttribute(Jav::Buffer &response,Net &store);
};

void init_openssl_library();

#endif

Net.cpp

#include <Jav/error/error.h>
#include <Jav/string/compare.h>
#include <Jav/string/builder.h>
#include <Jav/string/charconv.h>
#include "Net.h"
#include <openssl/err.h>
extern "C" _CRTIMP int __cdecl __MINGW_NOTHROW  _fileno (FILE*);
#include <openssl/applink.c> //essential to avoid no applink runtime error


void init_openssl_library()
{
 SSL_library_init();
 SSL_load_error_strings();
 OPENSSL_config(NULL);
}

bool parseUrl(Url &obj,const char *url)
{
 const char *scheme, *scheme_end;
 const char *host, *host_end;
 const char *path;

 if(url == NULL)
    throw Jav::Error("url is null");

 scheme = url;
 scheme_end = Jav::nextPosOf("://",scheme);
 if(scheme_end == NULL) throw Jav::Error("Url Error: invalid scheme");

 host = scheme_end + 3;
 path = host_end = Jav::nextPos('/',host);

 // Erase port number from host name
 host_end = Jav::nextPosOf(':',{host,host_end});
 if(host_end == NULL) host_end = path;

 obj.port = Jav::cstring(scheme,scheme_end);
 obj.host = Jav::cstring(host,host_end);
 obj.path = path;

 return true;
}


Net::Net(const char *url)
{
    parseUrl(this->url,url);

    const SSL_METHOD *method = SSLv23_method();
    if(method == NULL) on_openssl_fail(__LINE__);

    ctx = SSL_CTX_new(method);
    if(ctx == NULL) on_openssl_fail(__LINE__);

    // Cannot fail???
    //SSL_CTX_set_verify(ctx,SSL_VERIFY_PEER,verify_callback);
    //SSL_CTX_set_verify_depth(ctx,4);

    const long flags = SSL_OP_NO_SSLv2|SSL_OP_NO_SSLv3|SSL_OP_NO_COMPRESSION;
    SSL_CTX_set_options(ctx,flags);

    web = BIO_new_ssl_connect(ctx);
    if(web == NULL) on_openssl_fail(__LINE__);

    auto hostName = Jav::buildString("?:?",this->url.host,this->url.port);
    if(BIO_set_conn_hostname(web,hostName.c_str()) != 1)
        on_openssl_fail(__LINE__);

    BIO_get_ssl(web,&ssl);
    if(ssl == NULL) on_openssl_fail(__LINE__);

    const char *PREFERRED_CIPHERS = "HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4";
    if(SSL_set_cipher_list(ssl,PREFERRED_CIPHERS) != 1)
        on_openssl_fail(__LINE__);

    if(SSL_set_tlsext_host_name(ssl,this->url.host) != 1)
        on_openssl_fail(__LINE__);

    //out = BIO_new_fp(stdout,BIO_NOCLOSE);
    //if(out == NULL) on_openssl_fail(__LINE__);

    if(BIO_do_connect(web) != 1) on_openssl_fail(__LINE__);
    if(BIO_do_handshake(web) != 1) on_openssl_fail(__LINE__);

    X509 *cert = SSL_get_peer_certificate(ssl);
    if(cert == NULL) on_openssl_fail(__LINE__);
    X509_free(cert);

    //How to get the following verify method to not fail
    //if(SSL_get_verify_result(ssl) != X509_V_OK) on_openssl_fail(__LINE__);
}

Net::Net(Net &&rhs)
 : ctx(Jav::bitMove<SSL_CTX>(&rhs.ctx)),
   ssl(Jav::bitMove<SSL>(&rhs.ssl)),
   web(Jav::bitMove<BIO>(&rhs.web)),
   url(rhs.url)
{
}

Net& Net::operator=(Net &&rhs)
{
 Jav::bitMove<SSL_CTX>(&ctx,&rhs.ctx);
 Jav::bitMove<SSL>(&ssl,&rhs.ssl);
 Jav::bitMove<BIO>(&web,&rhs.web);
 url = rhs.url;
 return *this;
}

Net::~Net()
{
    //SSL_free(ssl);
    SSL_CTX_free(ctx);
    //BIO_free(out);
    BIO_free(web);
}

void Net::requestFile()
{
    auto cmd = Jav::buildString(
                    "GET ? HTTP/1.1\r\n"
                    "Host: ?\r\n"
                    "Connection: close\r\n\r\n",
                    url.path, url.host
                );

    if(BIO_write(web, cmd.c_str(), cmd.size()) <= 0)
        on_openssl_fail(__LINE__);
}

size_t Net::recieveResponse(Jav::Buffer &buf)
{
    size_t numRead = BIO_read(web, buf.end, buf.freeSpace());
    if(numRead < 0) on_openssl_fail(__LINE__);

    buf.end += numRead;
    return numRead;
}

void Net::on_openssl_fail(int line)
{
 ERR_print_errors_fp(stderr);
 throw Jav::Error("problem at line %i\n",line);
}


bool ServerResponseParser::parseStatusLine(Jav::Buffer &response,Net &store)
{
    void *section = &&PARSE_HTTP_VERSION;

    if(response.unread() == 0) goto STORE;

    PARSE_HTTP_VERSION:
    {
     const char *begin = (char*)response.pos;
     const char *end = Jav::nextPosOf(' ',{begin,(char*)response.end});
     if(end == NULL) goto STORE;

     if( !Jav::isMatch({begin,end},"HTTP") )
        throw Jav::Error("Expected HTTP version in status line of Server response");

     response.pos = (uint8_t*)end+1;
     section = &&PARSE_STATUS_CODE;
    }

    PARSE_STATUS_CODE:
    {
     const char *begin = (char*)response.pos;
     const char *end = Jav::nextPosOf(' ',{begin,(char*)response.end});
     if(end == NULL) goto STORE;

     if(end - begin != 3) throw Jav::Error("status code requires 3 digits: %i",end-begin);

     status = begin[0] == '2' ? STATUS_OK :
              begin[0] == '3' ? STATUS_REDIRECT :
              begin[0] == '4' ? STATUS_FAIL :
                /** ELSE */  STATUS_UNKNOWN;

     if(!std::isdigit(begin[1]) || !std::isdigit(begin[2]))
        throw Jav::Error("status code should be a number");

     response.pos = (uint8_t*)end+1;
     section = &&PARSE_STATUS_STRING;
    }

    PARSE_STATUS_STRING:
    {
     const char *begin = (char*)response.pos;
     const char *end = Jav::nextPosOf("\r\n",{(char*)response.pos,(char*)response.end});
     if(end == NULL) goto STORE;

     status_string = Jav::cstring(begin,end);
     response.pos = (uint8_t*)end+1;
    }

    if(status == STATUS_FAIL)
        throw Jav::Error("Request Failed: %s",status_string.str());

    if(status == STATUS_UNKNOWN)
        throw Jav::Error("Unsupported status code");

    return true;

    STORE:
    {
     if(response.pos == response.begin && !response.isEmpty())
        throw Jav::Error("Failed to parse status line of server response");

     shiftBytes(response,response.pos,response.begin);
     response.pos = response.begin;
     store.recieveResponse(response);

     if(response.isEmpty())
        throw Jav::Error("Failed to parse status line of server response");

     goto *section;
    }
}

bool ServerResponseParser::parseAttribute(Jav::Buffer &response,Net &store)
{
    enum { VAR_LOCATION=1, VAR_CONTENT_LENGTH, VAR_CONTENT_TYPE };

    void *section = &&PARSE_VARIABLE;
    int var;

    PARSE_VARIABLE:
    {
        const char *begin = (char*)response.pos;
        const char *end = Jav::nextPosOf(':',{begin,(char*)response.end});
        if(end == NULL) goto STORE;

        if( Jav::isMatch({begin,end},"Content-Length") ) var = VAR_CONTENT_LENGTH;
        else if( Jav::isMatch({begin,end},"Content-Type") ) var = VAR_CONTENT_TYPE;
        else if( Jav::isMatch({begin,end},"Location") ) var = VAR_LOCATION;
        else                                         var = 0;

        response.pos = (uint8_t*)skipSpaces(end+1);
        section = &&PARSE_VALUE;
    }

    PARSE_VALUE:
    {
        const char *begin = (char*)response.pos;
        const char *end = Jav::nextPosOf("\r\n",{begin,(char*)response.end});
        if(end == NULL) goto STORE;

        switch(var)
        {
         case VAR_LOCATION:
            if(status == STATUS_REDIRECT) content_url = Jav::cstring(begin,end);
            break;

         case VAR_CONTENT_LENGTH:
            if(status == STATUS_OK)
            if(!Jav::ston(begin,end,content_size,Jav::STON_UINT_DEC))
                throw Jav::Error("Content-Length is not a number");
            break;

         case VAR_CONTENT_TYPE:
            if(var == STATUS_OK)
            if(!Jav::isMatch({begin,end},"application/"))
                throw Jav::Error("mime type Expected: application/*, Found: %s",Jav::cstring(begin,end).str());
        }

        response.pos = (uint8_t*)end+2;
        return true;
    }

    STORE:
    {
     if(response.pos == response.begin && !response.isEmpty())
        throw Jav::Error("Failed to parse attribute of server response");

     shiftBytes(response,response.pos,response.begin);
     response.pos = response.begin;
     store.recieveResponse(response);

     if(response.isEmpty())
        throw Jav::Error("Failed to parse attribute of server response");

     goto *section;
    }
}

bool ServerResponseParser::parse(Jav::Buffer &response,Net &store)
{
    parseStatusLine(response,store);

    PARSE_ATTRIBUTES:
    while(true)
    {
        if(response.unread() == 0) goto STORE;

        if(Jav::isMatch(response.pos,"\r\n"))
        {
         response.pos += 2;
         return true;
        }

        parseAttribute(response,store);
    }

    STORE:
    {
     if(response.pos == response.begin && !response.isEmpty())
        throw Jav::Error("Failed to parse server response");

     shiftBytes(response,response.pos,response.begin);
     response.pos = response.begin;
     store.recieveResponse(response);

     if(response.isEmpty())
        throw Jav::Error("Failed to parse server response");

     goto PARSE_ATTRIBUTES;
    }
}

main.cpp

#include <Jav/error/error.h>
#include <Jav/console/screen.h>
#include <Jav/file/file.h>
#include "Net.h"

int main(int argc,const char **argv)
{
    Jav::Buffer buf(4096);

    init_openssl_library();
    Net net("https://www.apachefriends.org/xampp-files/7.4.27/xampp-windows-x64-7.4.27-2-VC15-installer.exe");
    net.requestFile();

    ServerResponseParser parser;
    parser.parse(buf,net);

    if(parser.status == STATUS_REDIRECT)
    {
     buf.clear();
     net = Net(parser.content_url);
     net.requestFile();
     parser.parse(buf,net);

     if(parser.status != STATUS_OK)
        throw Jav::Error("Server is playing games");
    }

    // How to deduce filename from the HTTP Server Response
    Jav::wFile file = Jav::createNewFile<Jav::wFile>("download.exe");

    Jav::ConsoleScreen screen = Jav::openTerminal<Jav::ConsoleScreen>();
    Jav::cstring progress(100);

    size_t totalWrit = 0;

    while(true)
    {
        if(buf.unread() == 0)
        {
         buf.clear();
         net.recieveResponse(buf);
         if(buf.isEmpty()) return 0;
        }

        size_t numWrit = file.write(buf.pos,buf.unread());
        totalWrit += numWrit;
        buf.pos += numWrit;

        Jav::Point screen_pos = getCursorPos(screen);
        Jav::Point screen_sz = getScreenSize(screen);

        auto sz = sprintf(progress,"%i/%i %i%%",totalWrit,parser.content_size,(int)(totalWrit*100.0/parser.content_size));
        memset(progress+sz,' ',screen_sz.x-sz);
        screen.write(progress,sz);
        setCursorPos(screen,0,screen_pos.y);
    }
}