Archive

Posts Tagged ‘boost’

Coroutines in C++/Boost

January 19th, 2016 No comments

Starting with 1.56, boost/asio provides asio::spawn() to work with coroutines. Just paste the sample code here, with minor modifications:

#include <boost/asio.hpp>
#include <boost/asio/spawn.hpp>
#include <boost/bind.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/enable_shared_from_this.hpp>
#include <iostream>
using namespace std;
using boost::asio::ip::tcp;


class session: public boost::enable_shared_from_this<session>
{
public:
    explicit session(boost::asio::io_service &io_service)
        : socket_(io_service), timer_(io_service), strand_(io_service)
    {
    }
    tcp::socket &socket()
    {
        return socket_;
    }
    void go()
    {
        boost::asio::spawn(strand_, boost::bind(&session::echo, shared_from_this(), _1));
        boost::asio::spawn(strand_, boost::bind(&session::timeout, shared_from_this(), _1));
    }
private:
    void echo(boost::asio::yield_context yield)
    {
        try {
            char data[128];
            while (true) {
                timer_.expires_from_now(boost::posix_time::seconds(10));
                size_t n = socket_.async_read_some(boost::asio::buffer(data), yield);
                boost::asio::async_write(socket_, boost::asio::buffer(data, n), yield);
            }
        } catch (exception &) {
            socket_.close();
            timer_.cancel();
        }
    }
    void timeout(boost::asio::yield_context yield)
    {
        while (socket_.is_open()) {
            boost::system::error_code ignored_ec;
            timer_.async_wait(yield[ignored_ec]);
            if (timer_.expires_from_now() <= boost::posix_time::seconds(0)) {
                socket_.close();
            }
        }
    }
    tcp::socket socket_;
    boost::asio::deadline_timer timer_;
    boost::asio::io_service::strand strand_;
};

void do_accept(boost::asio::io_service &io_service, unsigned short port, boost::asio::yield_context yield)
{
    tcp::acceptor acceptor(io_service, tcp::endpoint(tcp::v4(), port));
    while (true) {
        boost::system::error_code ec;
        boost::shared_ptr<session> new_session(new session(io_service));
        acceptor.async_accept(new_session->socket(), yield[ec]);
        if (!ec) {
            new_session->go();
        }
    }
}

int main()
{
    try {
        boost::asio::io_service io_service;
        boost::asio::spawn(io_service, boost::bind(do_accept, boost::ref(io_service), 2222, _1));
        io_service.run();
    } catch (exception &e) {
        cerr << "Exception: " << e.what() << endl;
    }
    return 0;
}

The Python in my previous article can be used to work with the code above. I also tried to write a TCP server with only boost::coroutines classes. select() is used, since I want the code to be platform independent. NOTE: with coroutines, we have only _one_ thread.

#ifdef _WIN32
#include <winsock2.h>
#include <ws2tcpip.h>
#include <windows.h>
#pragma comment(lib, "ws2_32.lib")
#pragma warning(disable: 4996)
#define sock_send(s, str, len)      send(s, str, len, 0)
#define sock_close(s)               closesocket(s)
#else
#include <arpa/inet.h>
#include <sys/types.h>
#include <sys/socket.h>
#define sock_send(s, str, len)      send(s, str, len, MSG_NOSIGNAL)
#define sock_close(s)               close(s)
#endif
#include <cerrno>
#include <cstdio>
#include <cstring>
#include <iostream>
#include <list>
#include <boost/bind.hpp>
#include <boost/coroutine/all.hpp>
#include <boost/shared_ptr.hpp>
using namespace std;


#ifdef _WIN32
struct Win32SocketWrapper
{
    Win32SocketWrapper()
    {
        WSADATA wsaData;
        WSAStartup(0x0202, &wsaData);
    }
    ~Win32SocketWrapper()
    {
        WSACleanup();
    }
} g_win32_socket_wrapper;
#endif


class session
{
    typedef boost::coroutines::symmetric_coroutine<void> coro_t;
public:
    explicit session(int sock)
        : socket_(sock)
    {
        echo_coro_ = coro_t::call_type(boost::bind(&session::echo, this, _1));
    }
    int socket()
    {
        return socket_;
    }
    void go()
    {
        echo_coro_();
    }
    void echo(coro_t::yield_type &yield)
    {
        int rc;
        char buffer[128];
        while (true) {
            memset(buffer, 0, sizeof(buffer));
            yield(); rc = recv(socket_, buffer, sizeof(buffer), 0);
            if (rc == 0 || rc == -1) { /* close or error */
                printf("socket[%d] closed, rc=%d..\n", socket_, rc);
                sock_close(socket_);
                socket_ = -1;
                /* do not release here, or the whole coroutine context will be invalid.. */
                break;
            } else {
                sock_send(socket_, buffer, rc);
            }
        }
    }
private:
    int socket_;
    coro_t::call_type echo_coro_;
};

void event_loop(int server_sock)
{
    list<boost::shared_ptr<session> > session_list;
    int rc, maxfd, client_sock;
    fd_set rdset;
    struct sockaddr_in client_addr;
    size_t addr_size = sizeof(struct sockaddr_in);

    while (true) {
        FD_ZERO(&rdset);
        FD_SET(server_sock, &rdset);
        maxfd = server_sock;
        list<boost::shared_ptr<session> >::iterator it = session_list.begin();
        while (it != session_list.end()) {
            if ((*it)->socket() == -1) {
                session_list.erase(it++);
            } else {
                FD_SET((*it)->socket(), &rdset);
                if (maxfd < (*it)->socket()) {
                    maxfd = (*it)->socket();
                }
                ++it;
            }
        }
        /* max fd value plus 1 */
        rc = select(maxfd+1, &rdset, 0, 0, NULL);
        if (rc == -1) {
            continue;
        } else {
            if (FD_ISSET(server_sock, &rdset)) {
                client_sock = (int)accept(server_sock, (struct sockaddr *)&client_addr, (socklen_t *)&addr_size);
                printf("socket[%d] accepted: %s:%d..\n", client_sock, inet_ntoa(client_addr.sin_addr), ntohs(client_addr.sin_port));
                boost::shared_ptr<session> new_session(new session(client_sock));
                new_session->go(); /* go first */
                session_list.push_back(new_session);
            }
            for (list<boost::shared_ptr<session> >::iterator it = session_list.begin(); it != session_list.end(); ++it) {
                if (FD_ISSET((*it)->socket(), &rdset)) {
                    (*it)->go();
                }
            }
        }
    }
}

int main() 
{
    int rc, server_sock;
    struct sockaddr_in server_addr;

    server_sock = (int)socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    server_addr.sin_port = htons(2222);
    rc = bind(server_sock, (struct sockaddr *)&server_addr, sizeof(struct sockaddr_in));
    if (rc < 0) {
        fprintf(stderr, "bind: %s.\n", strerror(errno));
        return -1;
    }
    listen(server_sock, 5);
    /* loop */
    event_loop(server_sock);
    sock_close(server_sock);
    return 0;
}
Categories: C/C++ Tags: , ,

Basic Usage of Boost MultiIndex Containers

December 28th, 2015 Comments off

Just take a simple note here.
The Boost Multi-index Containers Library provides a class template named multi_index_container which enables the construction of containers maintaining one or more indices with different sorting and access semantics.

#include <string>
#include <ostream>
#include <boost/multi_index_container.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/multi_index/identity.hpp>
#include <boost/multi_index/member.hpp>
#include <boost/multi_index/tag.hpp>
#include <boost/lambda/lambda.hpp>
using namespace std;
using namespace boost::multi_index;


struct auth
{
    string m_name;
    string m_pass;
    auth(const string &name, const string &pass)
        : m_name(name), m_pass(pass) { }
    bool operator<(const auth &o) const { return m_name < o.m_name; }
    friend ostream &operator<<(ostream &os, const auth &a) { os << "(" << a.m_name << ", " << a.m_pass << ")"; return os; }
};

struct employee
{
    int m_id;
    auth m_auth;
    int m_hire;
    int m_resign;
    employee(int id, const string &name, const string &pass, int hire, int resign)
        : m_id(id), m_auth(name, pass), m_hire(hire), m_resign(resign) { }
    bool operator<(const employee &o) const { return m_id < o.m_id; }
    friend ostream &operator<<(ostream &os, const employee &e) { os << "(" << e.m_id << ", " << e.m_auth << ", " << e.m_hire << ", " << e.m_resign << ")"; return os; }
};


struct auth_t { };

struct change_resign {
    int m_resign;
    change_resign(int r) : m_resign(r) { }
    void operator()(employee &e) { e.m_resign = m_resign; }
};

typedef multi_index_container<
    employee,
    indexed_by<
        /* sort by employee::operator< */
        ordered_unique<identity<employee> >,
        /* sort by less<int> on m_hire */
        ordered_non_unique<member<employee, int, &employee::m_hire> >,
        /* sort by less<auth> on m_auth */
        ordered_non_unique<tag<auth_t>, member<employee, auth, &employee::m_auth> >
    >
> employee_set;

int main()
{
    employee_set es;
    es.insert(employee(1, "555", "555pass", 2012, 0));
    es.insert(employee(2, "444", "444pass", 2011, 0));
    es.insert(employee(3, "333", "333pass", 2013, 0));
    es.insert(employee(4, "222", "222pass", 2015, 0));
    es.insert(employee(5, "555", "555pass", 2014, 0)); /* dup */
    typedef employee_set::nth_index<1>::type hire_index_t;
    typedef employee_set::index<auth_t>::type auth_index_t;
    cout << "Get a view to index #1 (m_hire).." << endl;
    hire_index_t &hire_index = es.get<1>();
    std::copy(hire_index.begin(), hire_index.end(), ostream_iterator<employee>(cout, "\n"));
    cout << "Get a view to index tag auth_t (m_auth).." << endl;
    const auth_index_t &auth_index = es.get<auth_t>();
    std::copy(auth_index.begin(), auth_index.end(), ostream_iterator<employee>(cout, "\n"));
    cout << "Find.." << endl;
    hire_index_t::iterator it = hire_index.find(2015);
#if 0
    employee t = *it;
    t.m_resign = 2048;
    hire_index.replace(it, t);
#else
    hire_index.modify_key(it, boost::lambda::_1=1111);
    int old_resign = it->m_resign;
    hire_index.modify(it, change_resign(2048), change_resign(old_resign));
#endif
    cout << (*it) << endl;
    cout << "Find all.." << endl;
    pair<auth_index_t::const_iterator, auth_index_t::const_iterator> pr = auth_index.equal_range(auth("555", ""));
    std::copy(pr.first, pr.second, ostream_iterator<employee>(cout, "\n"));
    return 0;
}

Output:

Get a view to index #1 (m_hire)..
(2, (444, 444pass), 2011, 0)
(1, (555, 555pass), 2012, 0)
(3, (333, 333pass), 2013, 0)
(5, (555, 555pass), 2014, 0)
(4, (222, 222pass), 2015, 0)
Get a view to index tag auth_t (m_auth)..
(4, (222, 222pass), 2015, 0)
(3, (333, 333pass), 2013, 0)
(2, (444, 444pass), 2011, 0)
(1, (555, 555pass), 2012, 0)
(5, (555, 555pass), 2014, 0)
Find..
(4, (222, 222pass), 1111, 2048)
Find all..
(1, (555, 555pass), 2012, 0)
(5, (555, 555pass), 2014, 0)

To use with pointer values, only limited change needed as highlighted:

#include <string>
#include <ostream>
#include <boost/multi_index_container.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/multi_index/identity.hpp>
#include <boost/multi_index/member.hpp>
#include <boost/multi_index/tag.hpp>
#include <boost/lambda/lambda.hpp>
using namespace std;
using namespace boost::multi_index;


struct auth
{
    string m_name;
    string m_pass;
    auth(const string &name, const string &pass)
        : m_name(name), m_pass(pass) { }
    bool operator<(const auth &o) const { return m_name < o.m_name; }
    friend ostream &operator<<(ostream &os, const auth &a) { os << "(" << a.m_name << ", " << a.m_pass << ")"; return os; }
};

struct employee
{
    int m_id;
    auth m_auth;
    int m_hire;
    int m_resign;
    employee(int id, const string &name, const string &pass, int hire, int resign)
        : m_id(id), m_auth(name, pass), m_hire(hire), m_resign(resign) { }
    bool operator<(const employee &o) const { return m_id < o.m_id; }
    friend ostream &operator<<(ostream &os, const employee *e) { os << "(" << e->m_id << ", " << e->m_auth << ", " << e->m_hire << ", " << e->m_resign << ")"; return os; }
};


struct auth_t { };

struct change_resign {
    int m_resign;
    change_resign(int r) : m_resign(r) { }
    void operator()(employee *e) { e->m_resign = m_resign; }
};

typedef multi_index_container<
    employee *,
    indexed_by<
        /* sort by employee::operator< */
        ordered_unique<identity<employee> >,
        /* sort by less<int> on m_hire */
        ordered_non_unique<member<employee, int, &employee::m_hire> >,
        /* sort by less<auth> on m_auth */
        ordered_non_unique<tag<auth_t>, member<employee, auth, &employee::m_auth> >
    >
> employee_set;

int main()
{
    employee_set es;
    es.insert(new employee(1, "555", "555pass", 2012, 0));
    es.insert(new employee(2, "444", "444pass", 2011, 0));
    es.insert(new employee(3, "333", "333pass", 2013, 0));
    es.insert(new employee(4, "222", "222pass", 2015, 0));
    es.insert(new employee(5, "555", "555pass", 2014, 0)); /* dup */
    typedef employee_set::nth_index<1>::type hire_index_t;
    typedef employee_set::index<auth_t>::type auth_index_t;
    cout << "Get a view to index #1 (m_hire).." << endl;
    hire_index_t &hire_index = es.get<1>();
    std::copy(hire_index.begin(), hire_index.end(), ostream_iterator<employee *>(cout, "\n"));
    cout << "Get a view to index tag auth_t (m_auth).." << endl;
    const auth_index_t &auth_index = es.get<auth_t>();
    std::copy(auth_index.begin(), auth_index.end(), ostream_iterator<employee *>(cout, "\n"));
    cout << "Find.." << endl;
    hire_index_t::iterator it = hire_index.find(2015);
#if 0
    employee *t = *it;
    t->m_auth.m_name = "888";
    /* must use replace() to notify changes in indexed fields */
    hire_index.replace(it, t);
#else
    hire_index.modify_key(it, boost::lambda::_1=1111);
    int old_resign = (*it)->m_resign;
    hire_index.modify(it, change_resign(2048), change_resign(old_resign));
#endif
    cout << (*it) << endl;
    cout << "Find all.." << endl;
    pair<auth_index_t::const_iterator, auth_index_t::const_iterator> pr = auth_index.equal_range(auth("555", ""));
    std::copy(pr.first, pr.second, ostream_iterator<employee *>(cout, "\n"));
    /* clear */
    for (employee_set::const_iterator it = es.begin(); it != es.end(); ++it) {
        delete *it;
    }
    employee_set().swap(es);
    return 0;
}
Categories: C/C++ Tags: ,

Spurious Wakeups

November 20th, 2014 No comments

http://vladimir_prus.blogspot.com/2005/07/spurious-wakeups.html

One of the two basic synchronisation primitives in multithreaded programming is called “condition variables”. Here’s a small example:

bool something_happened;
boost::mutex m;
boost::condition_variable c;
void thread1() {
    boost::mutex::scoped_lock(m);
    while (!something_happened) {
        c.wait(m);
    }
}
void thread2() {
    // do lots of work
    boost::mutex::scoped_lock(m);
    something_happened = true;
    c.notify_one();
}

Here, the call to “c.wait()” unlocks the mutex (allowing the other thread to eventually lock it), and suspends the calling thread. When another thread calls ‘notify’, the first thread wakes up, locks the mutex again (implicitly, inside ‘wait’), sees that variable is set to ‘true’ and goes on.

But why do we need the while loop, can’t we write:

    if (!something_happened) {
        c.wait(m);
    }

We can’t. And the killer reason is that ‘wait’ can return without any ‘notify’ call. That’s called spurious wakeup and is explicitly allowed by POSIX. Essentially, return from ‘wait’ only indicates that the shared data might have changed, so that data must be evaluated again.

Okay, so why this is not fixed yet? The first reason is that nobody wants to fix it. Wrapping call to ‘wait’ in a loop is very desired for several other reasons. But those reasons require explanation, while spurious wakeup is a hammer that can be applied to any first year student without fail.

The second reason is that fixing this is supposed to be hard. Most sources I’ve seen say that fixing that would require very large overhead on certain architectures. Strangely, no details were ever given, which made me wonder if avoiding spurious wakeups is simple, but all the threading experts secretly decided to tell everybody it’s hard.

After asking on comp.programming.thread, I at least know the reason for Linux (thanks to Ben Hutchings). Internally, wait is implemented as a call to the ‘futex’ system call. Each blocking system call on Linux returns abruptly when the process receives a signal — because calling signal handler from kernel call is tricky. What if the signal handler calls some other system function? And a new signal arrives? It’s easy to run out of kernel stack for a process. Exactly because each system call can be interrupted, when glibc calls any blocking function, like ‘read’, it does it in a loop, and if ‘read’ returns EINTR, calls ‘read’ again.

Can the same trick be used to conditions? No, because the moment we return from ‘futex’ call, another thread can send us notification. And since we’re not waiting inside ‘futex’, we’ll miss the notification(A third thread can get it, and change the value of predicate. — gonwan). So, we need to return to the caller, and have it reevaluate the predicate. If another thread indeed set it to true, we’ll break out of the loop.

So much for spurious wakeups on Linux. But I’m still very interested to know what the original reasons were.

==============================
Also see the explanation for spurious wakeups on the linux man page: pthread_cond_signal.
Last note: PulseEvent() in windows(manual-reset) = pthread_cond_signal() in linux, while SetEvent() in windows(auto-reset) = pthread_cond_broadcast() in linux, see here and here. And spurious wakeups are also possible on windows when using condition variables.

Categories: C/C++ Tags: ,

Exception Safety with shared_ptr

February 10th, 2014 No comments

Code snippet:

#include <iostream>
#include <boost/shared_ptr.hpp>

class A {
public:
    A() { std::cout << "in A::A()." << std::endl; }
    ~A() { std::cout << "in A::~A()." << std::endl; }
};

class B {
public:
    B() { std::cout << "in B::B()." << std::endl; throw 1024; }
    ~B() { std::cout << "in B::~B()." << std::endl; }
};

class C {
public:
    C() : m_a(new A), m_b(new B) { }
#ifndef _USE_SHARED_PTR
    ~C() { delete m_b; delete m_a; }
#endif
private:
#ifndef _USE_SHARED_PTR
    A *m_a;
    B *m_b;
#else
    boost::shared_ptr<A> m_a;
    boost::shared_ptr<B> m_b;
#endif
};

int main() {
    try { C c; } catch (...) { }
    return 0;
}

Output:

binson@binson-precise:~$ g++ ptr.cpp -o ptr
binson@binson-precise:~$ ./ptr
in A::A().
in B::B().
binson@binson-precise:~$ g++ -D_USE_SHARED_PTR ptr.cpp -o ptr
binson@binson-precise:~$ ./ptr
in A::A().
in B::B().
in A::~A().

Exception safety is ensured, when using shared_ptr. Memory allocated by m_a is freed even when an exception is thrown. The trick is: the destructor of class shared_ptr is invoked after the destructor of class C.

Categories: C/C++ Tags: ,

Writing UTF-8 String Using ofstream in C++

August 21st, 2013 No comments

I’ve googled a lot to find the answer. But none really solve the problem simply and gracefully, even on stackoverflow. So we’ll do ourselves here πŸ™‚

Actually, std::string supports operation using multibytes characters. This is the base of our solution:

static const char g_cs[] = "\xE4\xBD\xA0\xE5\xA5\xBD";

bool test_std_string()
{
    ofstream ofs("a.txt");
    ofs << string(g_cs) << endl;
    ofs.close();
    string s;
    ifstream ifs("a.txt");
    ifs >> s;
#if _WIN32
    wstring ws = utf8_to_ucs2(s);
    MessageBoxW(NULL, ws.c_str(), L"test_std_string", MB_OK);
#else
    cout << s << endl;
#endif
    return true;
}

g_cs is a Chinese word(“δ½ ε₯½” which means hello) encoded in UTF-8. The code works under both Windows(WinXP+VS2005) and Linux(Ubuntu12.04+gcc4.6). You may wanna open a.txt to check whether the string is correctly written.

NOTE: Under Linux, we print the string directly since the default console encoding is UTF-8, and we can view the string. While under Window, the console DOES NOT support UTF-8(codepage 65001) encoding. Printing to it simply causes typo. We just convert it to a std::wstring and use MessageBox() API to check the result. I will cover the encoding issue in windows console in my next post, maybe.

I began to investigate the problem, since I cannot find a solution to read/write a UTF-8 string to XML file using boost::property_tree. Actually, it’s a bug and is already fixed in boost 1.47 and later versions. Unfortunately, Ubuntu 12.04 came with boost 1.46.1. When reading non-ASCII characters, some bytes are incorrectly skipped. The failure function is boost::property_tree::detail::rapidxml::internal::get_index(). My test code looks like:

static const char g_xml[] = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"
    "<aaa>\n"
        "<bbb>\xE4\xBD\xA0\xE5\xA5\xBD</bbb>\n"
        "<bbb>\xE7\xA5\x9E\xE9\xA9\xAC</bbb>\n"
    "</aaa>\n";

bool test_boost_ptree()
{
    /* write to file */
    FILE *f = fopen("a.xml", "w");
    fwrite(g_xml, sizeof(g_xml)-1, 1, f);
    fclose(f);
    /* read and modify */
    const char cstr[] = "\xE4\xB8\xB8\xE5\xAD\x90\xE9\x85\xB1";
    try {
        boost::property_tree::ptree pt;
        int flags = boost::property_tree::xml_parser::trim_whitespace;
        boost::property_tree::read_xml("a.xml", pt, flags, std::locale());
        boost::property_tree::ptree pt2 = pt.get_child("aaa").add("bbb", string(cstr));
        boost::property_tree::xml_writer_settings<char> settings(' ', 2);
        boost::property_tree::write_xml("b.xml", pt, std::locale(), settings);
    } catch (boost::property_tree::xml_parser_error &) {
        return false;
    } catch (boost::property_tree::ptree_bad_path &) {
        return false;
    }
    /* read again */
    try {
        boost::property_tree::ptree pt;
        int flags = boost::property_tree::xml_parser::trim_whitespace;
        boost::property_tree::read_xml("b.xml", pt, flags, std::locale());
        string s = pt.get<string>("aaa.bbb");
#if _WIN32
        wstring ws = utf8_to_ucs2(s);
        MessageBoxW(NULL, ws.c_str(), L"test_boost_ptree", MB_OK);
#else
        cout << s << endl;
#endif
    } catch (boost::property_tree::xml_parser_error &) {
        return false;
    } catch (boost::property_tree::ptree_bad_path &) {
        return false;
    }
    return true;
}

Almost the same structure with the previous function. And finally the utf8_to_ucs2() function:

#ifdef _WIN32
wstring utf8_to_ucs2(const string &input)
{
    wchar_t *pwc;
    wstring output;
    int len = MultiByteToWideChar(CP_UTF8, 0, input.c_str(), (int)input.length(), NULL, 0);
    pwc = new wchar_t[len+1];
    ZeroMemory(pwc, sizeof(wchar_t)*(len+1));
    MultiByteToWideChar(CP_UTF8, 0, input.c_str(), (int)input.length(), pwc, len+1);
    output = pwc;
    delete pwc;
    return output;
}
#endif

Please add header files yourselves to make it compile πŸ™‚

Categories: C/C++ Tags: , ,

Logging in Multithreaded Environment Using Thread-Local Storage

April 12th, 2013 No comments

Generally, A logger is a singleton class. The declaration may look like:

#ifndef _LOGGER_H
#define _LOGGER_H

#include <string>

class Logger
{
private:
    Logger() { }
public:
    static void Init(const std::string &name);
    static Logger *GetInstance();
    void Write(const char *format, ...);
private:
    static std::string ms_name;
    static Logger *ms_this_logger;
};

#endif

The Init function is used to set log name or maybe other configuration information. And We can use the Write function to write logs.

Well, in a multithreaded environment, locks must be added to prevent concurrent issues and keep the output log in order. And sometimes we want to have separate log configurations. How can we implement it without breaking the original interfaces?

One easy way is to maintain a list of all available Logger instances, so that we can find and use a unique Logger in each thread. The approach is somehow like the one used in log4j. But log4j reads configuration files to initialize loggers, while our configuration information is set in runtime.

Another big issue is that we must add a new parameter to the GetInstance function to tell our class which Logger to return. The change breaks interfaces.

By utilizing TLS (thread-local storage), we can easily solve the above issues. Every logger will be thread-local, say every thread has its own logger instance which is stored in its thread context. Here comes the declaration for our new Logger class, boost::thread_specific_ptr from boost library is used to simplify our TLS operations:

#ifndef _LOGGER2_H
#define _LOGGER2_H

#include <string>
#include <boost/thread.hpp>

class Logger
{
private:
    Logger() { }
public:
    static void Init(const std::string &name);
    static Logger *GetInstance();
    void Write(const char *format, ...);
private:
    static boost::thread_specific_ptr<std::string> ms_name;
    static boost::thread_specific_ptr<Logger> ms_this_logger;
};

#endif

Simply use boost::thread_specific_ptr to wrap the original 2 static variables, and they will be in TLS automatically, that’s all. The implementation:

#include "logger2.h"
#include <stdio.h>
#include <stdarg.h>
#include <string.h>

using namespace std;

boost::thread_specific_ptr<string> Logger::ms_name;
boost::thread_specific_ptr<Logger> Logger::ms_this_logger;

void Logger::Init(const string &name)
{
    if (!name.empty()) {
        ms_name.reset(new std::string(name));
    }
}

Logger *Logger::GetInstance()
{
    if (ms_this_logger.get() == NULL) {
        ms_this_logger.reset(new Logger);
    }
    return ms_this_logger.get();
}

void Logger::Write(const char *format, ...)
{
    va_list arglist;
    char buffer[1024];
    va_start(arglist, format);
    memset(buffer, 0, sizeof(buffer));
    vsnprintf(buffer, sizeof(buffer), format, arglist);
    va_end(arglist);
    printf("[%s] %s\n", ms_name.get()->c_str(), buffer);
}

Our test code:

#include <boost/date_time.hpp>
#include <boost/thread.hpp>
/*
 * actually, we do not matter which header file to include,
 * since they have compatible public interface, compatible ABI.
 */
//#include "logger.h"
#include "logger2.h"

using namespace std;

class Thread
{
public:
    Thread(const char *name) : m_name(name) { }
    void operator()()
    {
        /* set logger name in thread */
        Logger::Init(m_name);
        /* call GetInstance() and Write() in other functions with thread-local enabled */
        Logger *logger = Logger::GetInstance();
        for (int i = 0; i < 3; i++) {
            logger->Write("Hello %d", i);
            boost::this_thread::sleep(boost::posix_time::seconds(1));
        }
    }
private:
    string m_name;
};

int main()
{
    boost::thread t1(Thread("name1"));
    boost::thread t2(Thread("name2"));
    t1.join();
    t2.join();
    return 0;
}

Output when using the original Logger may look like:

[name1] Hello 0
[name2] Hello 0
[name2] Hello 1
[name2] Hello 1
[name2] Hello 2
[name2] Hello 2

When using the TLS version, it may look like:

[name1] Hello 0
[name2] Hello 0
[name1] Hello 1
[name2] Hello 1
[name1] Hello 2
[name2] Hello 2

Everything is in order now. You may want to know what OS API boost uses to achieve TLS. I’ll show you the details in boost 1.43:

# windows implementation
boost::thread_specific_ptr::reset()
  --> boost::detail::set_tss_data()
  --> boost::detail::get_or_make_current_thread_data()
  --> boost::detail::get_current_thread_data()
  --> ::TlsGetValue()
# see:
# ${BOOST}/boost/thread/tss.hpp
# ${BOOST}/lib/thread/src/win32/thread.cpp
# *nix implementation
boost::thread_specific_ptr::reset()
  --> boost::detail::set_tss_data()
  --> boost::detail::add_new_tss_node()
  --> boost::detail::get_or_make_current_thread_data()
  --> boost::detail::get_current_thread_data()
  --> ::pthread_getspecific()
# see:
# ${BOOST}/boost/thread/tss.hpp
# ${BOOST}/lib/thread/src/pthread/thread.cpp

The underlying API is TlsGetValue under windows and pthread_getspecific under *nix platforms.

Categories: C/C++ Tags: , ,

Smart Pointers in C++0x and Boost (2)

August 14th, 2011 No comments

1. Environment

– windows xp
– gcc-4.4
– boost-1.43

2. auto_ptr

A smart pointer is an abstract data type that simulates a pointer while providing additional features, such as automatic garbage collection or bounds checking. There’s auto_ptr in C++03 library for general use. But it’s not so easy to deal with it. You may encounter pitfalls or limitations. The main drawback of auto_ptr is that it has the transfer-of-ownership semantic. I just walk through it. Please read comments in code carefully:

int *test_auto_ptr_exp() {
    auto_ptr<int> p(new int(1));
    throw runtime_error("auto_ptr test exception.");
    /* exception-safe, p is free even when an exception is thrown. */
    return p.get();
}

void test_auto_ptr_basic() {
    auto_ptr<int> p1(new int(1));
    auto_ptr<int> p2(new int(2));
    auto_ptr<int> p3(p1);
    auto_ptr<int> p4;
    p4 = p2;
    if (p1.get()) {  /* NULL */
        cout << "*p1=" << *p1 << endl;
    }
    if (p2.get()) {  /* NULL */
        cout << "*p2=" << *p2 << endl;
    }
    if (p3.get()) {  /* ownership already transferred from p1 to p3 */
        cout << "*p3=" << *p3 << endl;
    }
    if (p4.get()) {  /* ownership already transferred from p2 to p4 */
        cout << "*p4=" << *p4 << endl;
    }
    /* ERROR: void is a type of template specialization */
    //auto_ptr<void> ptr5(new int(3));
}

void test_auto_ptr_errors() {
    /* ERROR: statically allocated object */
    const char *str = "Hello";
    auto_ptr<const char> p1(str);
    /* ERROR: two auto_ptrs refer to the same object */
    int *pi = new int(5);
    auto_ptr<int> p2(pi);
    auto_ptr<int> p3(p2.get());
    p2.~auto_ptr();  /* now p3 is not available too */
    /* ERROR: hold a pointer to a dynamically allocated array */
    /* When destroyed, it only deletes first single object. */
    auto_ptr<int> (new int[10]);
    /* ERROR: store an auto_ptr in a container */
    //vector<auto_ptr<int> > vec;
    //vec.push_back(auto_ptr<int>(new int(1)));
    //vec.push_back(auto_ptr<int>(new int(2)));
    //auto_ptr<int> p4(vec[0]);  /* vec[0] is assigned NULL */
    //auto_ptr<int> p5;
    //p5 = vec[1];  /* vec[1] is assigned NULL */
}

3. unique_ptr

To resolve the drawbacks, C++0x deprecates usage of auto_ptr, and unique_ptr is the replacement. unique_ptr makes use of a new C++ langauge feature called rvalue reference which is similar to our current (left) reference (&), but spelled (&&). GCC implemented this feature in 4.3, but unique_ptr is only available begin from 4.4.

What is rvalue?

rvalues are temporaries that evaporate at the end of the full-expression in which they live (“at the semicolon”). For example, 1729, x + y, std::string(“meow”), and x++ are all rvalues.

While, lvalues name objects that persist beyond a single expression. For example, obj, *ptr, ptr[index], and ++x are all lvalues.

NOTE: It’s important to remember: lvalueness versus rvalueness is a property of expressions, not of objects.

We may have another whole post to address the rvalue feature. Now, let’s take a look of the basic usage. Please carefully reading the comments:

unique_ptr<int> get_unique_ptr(int i) {
    return unique_ptr<int> (new int(i));
}

void use_unique_ptr(unique_ptr<int> p) {
    /* p is deleted when finish running this function. */
}

void test_unique_ptr_basic() {
    unique_ptr<int> p(new int(1));
    /*
     * One can make a copy of an rvalue unique_ptr.
     * But one can not make a copy of an lvalue unique_ptr.
     * Note the defaulted and deleted functions usage in source code(c++0x).
     */
    //unique_ptr<int> p2 = p;  /* error */
    //use_unique_ptr(p);       /* error */
    use_unique_ptr(move(p));
    use_unique_ptr(get_unique_ptr(3));
}

One can ONLY make a copy of an rvalue unique_ptr. This confirms no ownership issues occur like that of auto_ptr. Since temporary values cannot be referenced after the current expression, it is impossible for two unique_ptr to refer to a same pointer. You may also noticed the move function. We will also discuss it in a later post.

Some more snippet:

struct aclass {
    aclass() { cout << "in aclass::ctor()" << endl; }
    ~aclass() { cout << "in aclass::dtor()" << endl; }
};

struct aclass_deleter {
    void operator()(void *p) {
        delete static_cast<aclass *> (p);
    }
};

template<class T>
struct array_deleter {
    void operator()(T *array) {
        delete[] array;
    }
};

typedef array_deleter<aclass> aclass_array_deleter;
typedef unique_ptr<aclass, aclass_array_deleter> aclass_array_ptr;

void test_unique_ptr_custom_deleter() {
    /*
     * Hold a pointer to a dynamically allocated array.
     * aaptr & aaptr2 are deleted when finish running this function.
     */
    aclass_array_ptr aaptr(new aclass[3]);
    unique_ptr<aclass[]> aaptr2(new aclass[3]);  /* default_deleter<T[]> */
    /* allow void pointer, but a custom deleter must be used. */
    unique_ptr<void, aclass_deleter> p3(new aclass);
}

unique_ptr can hold pointers to an array. unique_ptr defines deleters to free memory of its internal pointer. There are pre-defined default_deleter using delete and delete[](array) for general deallocation. You can also define your customized ones. In addition, a void type can be used.

NOTE: To compile the code, you must specify the -std=c++0x flag.

4. shared_ptr

A shared_ptr is used to represent shared ownership; that is, when two pieces of code needs access to some data but neither has exclusive ownership (in the sense of being responsible for destroying the object). A shared_ptr is a kind of counted pointer where the object pointed to is deleted when the use count goes to zero.

Following snippet shows the use count changes when using shared_ptr. The use count changes from 0 to 3, then changes back to 0:

struct bclass {
    int i;
    bclass(int i) { this->i = i; }
    virtual ~bclass() { cout << "in bclass::dtor() with i=" << i << endl; }
};

struct cclass: bclass {
    cclass(int i) : bclass(i) { }
    virtual ~cclass() { cout << "in cclass::dtor() with i=" << i << endl; }
};

void use_shared_ptr(shared_ptr<int> p) {
    cout << "count=" << p.use_count() << endl;
}

void test_shared_ptr_basic() {
    shared_ptr<int> p;
    cout << "count=" << p.use_count() << endl;
    p.reset(new int(1));
    cout << "count=" << p.use_count() << endl;
    shared_ptr<int> p2 = p;
    cout << "count=" << p.use_count() << endl;
    use_shared_ptr(p2);
    cout << "count=" << p.use_count() << endl;
    p2.~shared_ptr();
    cout << "count=" << p.use_count() << endl;
    p2.~shared_ptr();
    cout << "count=" << p.use_count() << endl;
}

Snippets showing pointer type conversion:

void test_shared_ptr_convertion() {
    /* p is deleted accurately without custom deleter */
    shared_ptr<void> p(new aclass);
    /* use parent type to hold child object */
    shared_ptr<bclass> p2(new cclass(10));
    shared_ptr<cclass> p3 = static_pointer_cast<cclass> (p2);
    cout << "p3->i=" << p3->i << endl;
    p3->i = 20;
    cout << "p2->i=" << p2->i << endl;
}

The void type can be used directly without a custom deleter, which is required in unique_ptr. Actually, shared_ptr has already save the exact type info in its constructor. Refer to source code for details :). And static_pointer_cast function is used to convert between pointer types.

Unlike auto_ptr, Since shared_ptr can be shared, it can be used in STL containers:

typedef shared_ptr<bclass> bclass_ptr;

struct bclass_ops {
    void operator()(const bclass_ptr&amp; p) {
        cout << p->i << endl;
    }
    bool operator()(const bclass_ptr& a, const bclass_ptr& b) {
        return a->i < b->i;
    }
};

void test_shared_ptr_containers() {
    vector<bclass_ptr> vec1, vec2;
    bclass_ptr ptr(new bclass(1));
    vec1.push_back(ptr);
    vec2.push_back(ptr);
    ptr.reset(new bclass(2));
    vec1.push_back(ptr);
    vec2.push_back(ptr);
    ptr.reset(new bclass(3));
    vec1.push_back(ptr);
    vec2.push_back(ptr);
    for_each(vec1.begin(), vec1.end(), bclass_ops());
    reverse(vec2.begin(), vec2.end());
    for_each(vec2.begin(), vec2.end(), bclass_ops());
}

NOTE: shared_ptr is available in both TR1 and Boost library. You can use either of them, for their interfaces are compatible. In addition, there are dual C++0x and TR1 implementation. The TR1 implementation is considered relatively stable, so is unlikely to change unless bug fixes require it.

5. weak_ptr

weak_ptr objects are used for breaking cycles in data structures. See snippet:

struct mynode {
    int i;
    shared_ptr<mynode> snext;
    weak_ptr<mynode> wnext;
    mynode(int i) { this->i = i; }
    ~mynode() { cout << "in mynode::dtor() with i=" < i < endl; }
};

void test_weak_ptr() {
    shared_ptr<mynode> head(new mynode(1));
    head->snext = shared_ptr<mynode>(new mynode(2));
    /* use weak_ptr to solve cyclic dependency */
    //head->snext = head;
    head->wnext = head;
}

If we use uncomment to use shared_ptr, head is not freed since there still one reference to it when exiting the function. By using weak_ptr, this code works fine.

6. scoped_ptr

scoped_ptr template is a simple solution for simple needs. It supplies a basic “resource acquisition is initialization” facility, without shared-ownership or transfer-of-ownership semantics.

This class is only available in Boost. Since unique_ptr is already there in C++0x, this class may be thought as redundant. Snippet is also simple:

void test_scoped_ptr() {
    /* simple solution for simple needs */
    scoped_ptr<aclass> p(new aclass);
}

Complete and updated code can be found on google code host here. I use conditional compilation to swith usage between TR1 and Boost implementation in code. Hope you find it useful.

Categories: C/C++ Tags: ,

Smart Pointers in C++0x and Boost

August 14th, 2011 No comments

Let clarify some concepts first. What is C++0x? Wikipedia gives some overview here:

C++0x is intended to replace the existing C++ standard, ISO/IEC 14882, which was published in 1998 and updated in 2003. These predecessors are informally but commonly known as C++98 and C++03. The new standard will include several additions to the core language and will extend the C++ standard library, incorporating most of the C++ Technical Report 1 (TR1) libraries β€” with the exception of the library of mathematical special functions.

Then why it is called C++0x? As Bjarne Stroustrup addressed here:

The aim is for the ‘x’ in C++0x to become ‘9’: C++09, rather than (say) C++0xA (hexadecimal :-).

You may also noticed TR1, also refer here in Wikipedia:

C++ Technical Report 1 (TR1) is the common name for ISO/IEC TR 19768, C++ Library Extensions, which is a document proposing additions to the C++ standard library. The additions include regular expressions, smart pointers, hash tables, and random number generators. TR1 is not a standard itself, but rather a draft document. However, most of its proposals are likely to become part of the next official standard.

You got the relationship? C++0x is the standard adding features to both language and standard library. A large set of TR1 libraries and some additional libraries. For instance, unique_ptr is not defined in TR1, but is included in C++0x.

As of 12 August 2011, the C++0x specification has been approved by the ISO.

Another notable concept is the Boost library. It can be regarded as a portable, easy-to-use extension to the current C++03 standard library. And some libraries like smart pointers, regular expressions have already been included in TR1. You can find license headers regarding the donation of the boost code in libstdc++ source files. While in TR2, some more boost code are to be involved.

TR1 libraries can be accessed using std::tr1 namespace. More info on Wikipedia here:

Various full and partial implementations of TR1 are currently available using the namespace std::tr1. For C++0x they will be moved to namespace std. However, as TR1 features are brought into the C++0x standard library, they are upgraded where appropriate with C++0x language features that were not available in the initial TR1 version. Also, they may be enhanced with features that were possible under C++03, but were not part of the original TR1 specification.

The committee intends to create a second technical report (called TR2) after the standardization of C++0x is complete. Library proposals which are not ready in time for C++0x will be put into TR2 or further technical reports.

The article seems to be a bit too long so far, I decide to give my snippets in a later post.

Categories: C/C++ Tags: ,