Copyright © 2001 - 2007 SILC Project
SILC Project Website
SILC Toolkit Reference Manual
Index

SILC Toolkit Reference Manual
SILC Crypto Library
    Introduction to SILC RNG
    SILC RNG Interface
    SILC Cipher API
    SILC PKCS API
    SILC Public Key API
    SILC PKCS #1 API
    SILC Hash Interface
    SILC HMAC Interface
SILC Core Library
    SILC Authentication Interface
    SILC Message Interface
    SILC Channel Interface
    SILC Command Interface
    SILC Notify Interface
    SILC Status Types
    SILC Modes
    SILC ID Interface
    SILC Argument Interface
    SILC Attributes Interface
    Packet Engine Interface
    SILC Public Key Payload Interface
SILC Key Exchange Library
    SILC SKE Interface
    SILC Connection Authentication Interface
SILC VCard Library
    SILC VCard Interface
SILC Math Library
    SILC MP Interface
    SILC Math Interface
SILC Client Library
    Using SILC Client Library Tutorial
    Arguments for command_reply Client Operation
    SilcStatus Error Arguments in command_reply Client Operation
    Arguments for notify Client Operation
    Unicode and UTF-8 Strings in Client Library
    Client Library Interface Reference
    Client Entry Interface Reference
SILC ASN.1 Library
    SILC ASN.1 Interface
    SILC BER interface
SILC HTTP Library
    SILC HTTP Server Interface
    SILC HTTP PHP Translator
SILC Utility Library
    Basic Types and Definitions
    Data Buffer Interface
    Data Buffer Format Interface
    Hash Table Interface
    Memory Allocation Interface
    Data Stack (memory pool) Interface
    Finite State Machine Interface
    Thread Interface
    Mutual Exclusion Lock Interface
    Condition Variable Interface
    Atomic Operations Interface
    Network (TCP and UDP) Interface
    Scheduler Interface
    Asynchronous Operation Interface
    Abstract Stream Interface
    Socket Stream Interface
    File Descriptor Stream Interface
    File Utility Functions
    String Utility Interface
    Snprintf Interface
    UTF-8 String Interface
    Stringprep Interface
    Utility Functions
    List Interface
    Dynamic List Interface
    MIME Interface
    Time Utility Functions
    Logging Interface
    Config File Interface
SILC Key Repository Library
    SILC SKR Interface
SILC Application Utility Library
    SILC Application Utilities
    SILC ID Cache Interface
SILC SFTP Library
    SILC SFTP Interface
    SFTP Filesystems Interface

Resource Links
SILC Project Website
SILC Protocol Documentation
SILC White Paper
SILC FAQs





Porting from Toolkit 1.0 to 1.1
 
This documents describes the differences between the SILC Toolkit 1.0 and 1.1 and should help application programmers to port their SILC applications to the new Toolkit version.
 
  • General
  • Platform changes
  • Client library
  • Utility library
  • Application utility library
  • Crypto library
  • Key Repository library
  • VCard library
  • HTTP library
  • ASN.1 library
     

    General changes

    The main SILC Toolkit header file has changed its name from silcincludes.h to silc.h. The first task in porting from 1.0 to 1.1 is to change these filenames in your source tree.
     

    Platform changes

    The Windows support has been made better by improving the SILC scheduler and network routines. Calling silc_net_win32_init and silc_net_win32_uninit is not needed anymore. The network routines on Windows are enabled automatically.
     

    Client library, lib/silcclient/

    The client library in Toolkit 1.1 has been partially rewritten. It was rewritten to work out some technical issues and to increase performance and decrease memory consumption. The client library API has also faced some changes, altough most of it has remained the same. Most major change in the client library is that is now fully supports multiple threads. In the past making multithreaded SILC client application was always a bit hard but with Toolkit 1.1 it should be relatively straightforward.

    SilcClientOperations structure

    The SilcClientOperations structure has changed significantly. It no longer has `connected', `disconnected' and `failure' function pointers. Instead a new SilcClientConnectCallback has been defined that is given as argument to functions like silc_client_connect_to_server, silc_client_connect_to_client and silc_client_key_exchange (last two being entirely new functions in the API). That callback is called when connection is established and when it is disconnected, or if a protocol failure occurred during connecting. That callback also delivers a detailed error information when error occurs. New connection related status types in SilcClientConnectionStatus has also been added, see the API for the details.
     
    Also the `detach' client operation function pointer has been removed. Instead the detachment data is now simply delivered in the SILC_COMMAND_DETACH command reply.
     
    Some of the remaining client operation function pointers have changed a bit. Most major change is the `command' operation now having the arguments that user gave on the user interface sent back to application. The `command_reply' does not anymore include the SilcCommandPayload as an argument. Also the variable argument lis is now delivered as va_list. Also the boolean success indicator is removed and SilcStatus error argument instead is telling the exact error if the SilcStatus status is other than SILC_STATUS_OK.
     
    The `get_auth_method' client operation now includes the authentication method that has been resolved from the remote server. The application no longer needs to resolve it and the function silc_client_request_authentication_method has been removed from the API.
     
    The `verify_public_key' client operation now include the SilcPublicKey pointer instead of the public key in raw encoded form. Also the connection type argument has changed to SilcConnectionType.
     
    The `key_agreement' client operation now includes a new protocol argument which tells whether the key agremeent is supposed to be done on TCP or on UDP transport. The new Toolkit 1.1 supports both.

    SilcClient

    The SilcClientParams includes new fields and some fields have been removed or moved to other places. All connection timeout related fields have been moved to SilcClientConnectionParams. The nickname_parse callback pointer has also been removed from SilcClientParams. The Toolkit 1.1 now automatically parses formatted nicknames.
     
    The new field boolean `threads' can be used to tell whether the new SilcClient is to use multiple threads or not. If threads support has been compiled in and that field is set to TRUE then the client library will create new thread for each new connection to a remote host. If you are going to use several connections or some of the connections are high throughput connections (like multimedia connections) then using threads is recommended for performance. Note that your application needs to be multithreaded and, if necessary to, perform concurrencly control in client operation callbacks which may be called from multiple threads at the same time for different connections. However, it is always guaranteed that for one connection the client operations cannot be called from multiple threads. The multithreads support in Toolkit 1.1 client library means that each new connection and everything related to that connection is handled in own dedicated thread.
     
    The silc_client_init function now takes several new arguments. It now takes the username, hostname and realname as argument that used to be required by the application to update directly to SilcClient pointer. This is not longer required. Also the nickname is no longer required to be put to SilcClient context as it has been made SilcClientConnection specific. The function also takes a SilcClientRunning callback as argument which will be called when the client library is running. Only after the callback is delivered may the application start using other client library API functions. For example, connecting to a server will fail if it is performed before the SilcClientRunning callback is called. The callback is called after the silc_client_run or silc_client_run_one has been called.
     
    The silc_client_stop function takes new SilcClientStopped callback as argument, and it will be called after the client library has been completely stopped. The application should not quit before that is called in order to clean up all resources allocation by the client library.
     
    It is also not required to put the public key pair (SilcPublicKey and SilcPrivateKey) into SilcClient. They have been made connection specific so you simply give them as argument when you create new connection. This way it is now possible to use different key pairs with different connections, one that was not possible in Toolkit 1.0 without creating new SilcClient instance for each new connection.

    SilcClientConnection

    The SilcClientConnection represents connection to a remote host. In new Toolkit 1.1 this structure now includes all connection related data and no connection related data is anymore included in SilcClient. For this reason it is no longer necessary to create multiple SilcClient instances just because you want to create multiple connections.
     
    The SilcClientConnectionParams structure has many new fields. This structure is given as argument to any function that is able to create a new connection or to create a network listener or to create a new SILC sessions. See the API documentation for all the details but the following one field of special relevance when creating new SILC server connections. The nickname field is now included in this structure and it is the nickname user would like to initially use in the SILC network.
     
    The functions silc_client_connect_to_server, silc_client_connect_to_client and silc_client_key_exchange can all be used to create new SILC session with a remote. To connect SILC server use silc_client_connect_to_server. You now give the SilcClientConnectionParams as argument which includes the nicknameuser wants to use, you now can give the public key pair as argument that is to be used in the connecting. The new SilcClientConnectCallback will be called once the connection has been established and it delivers the SilcClientConection context to the application. It will be later called again to indicated disconnection from the remote host. These functions now return SilcAsyncOperation context which can be used to cancel the connecting, if necessary. In the past canceling connection was not possible.
     
    SilcAsyncOperation op;
    SilcClientConnectionParams params;
    
    memset(¶ms, 0, sizeof(params));
    
    /* Set nickname user wants to use */
    params.nickname = nickname;
    
    /* Set perfect forward secery for key exchange */
    params.pfs = TRUE;
    
    /* Example of creating, and then canceling a connection */
    op = silc_client_connect_to_server(client, params, public_key, private_key,
                                       remote_host, remote_port, connection_cb,
    				   app_context);
    if (!op) {
      fatal("Connecting failed immediately"));
      exit(1);
    }
    
    ...
    
    /* Cancel connecting.  After canceling the `connection_cb' will not be
       called. */
    silc_async_abort(op);
    

     
    The old function silc_client_start_key_exchange has been replaced with silc_client_key_exchange function. Semantically it is equivalent to silc_client_connect_to_server but it does not create new connection. Instead, it accepts SilcStream as argument which is already established connection to a remote host and it merely starts the key exchange. See an example in the API documentation on how to use the silc_client_key_exchange, if your application wishes itself to create connection instead of using the client library to do it. The functions silc_client_add_connection, silc_client_del_connection and silc_client_del_socket has been removed. They are no longer needed.

    Entries

    Just like in Toolkit 1.0 we now have SilcClientEntry to represent user, SilcChannelEntry to represent channel and SilcServerEntry to represent server. In the past these structures and all API functions that dealt with them were in silcclient.h file. They are now in silcclient_entry.h in Toolkit 1.1.
     
    As an general convention each of these new entries now are reference counted and they have locks when using them in multithreaded application. Even in one-threaded application the application must always acquire a reference of the entry if it wishes to save the entry pointer in the application. The reference must be released once the entry pointer is not needed anymore. This ensures that the library cannot free the entry pointer underneath the application. An own API for taking and releasing the reference is in Toolkit 1.1.
     
    /* Take reference of the client entry I save to my own context. */
    my_context->client_entry = silc_client_ref_client(client, conn, client_entry);
    

     
    If multiple threads are used and application wants to access the entry it must always first lock the entry. After application has read the information it needs from the entry it must release the entry lock. The entry lock should be held only for short periods of time and failure to release the lock will result into deadlock. An own API for taking and releasing the entry lock is in Toolkit 1.1. If you application is not multithreaded you do not need to use the entry locking.
     
    /* Read data from client entry in multithreaded environment */
    silc_client_lock_client(client_entry);
    fprintf(stdout, "%s\n", client_entry->nickname);
    fprintf(stdout, "%s\n", silc_id_render(SILC_ID_CLIENT, &client_entry->id));
    silc_client_unlock_client(client_entry);
    

     
    From the entries all unnecessary information for application has been either removed or moved to internal structure that is not accessible by the application. As a result the entry structures are much smaller and cleaner.
     
    The SilcClientEntry now includes nickname_normalized field and it is the normalized version of the user's nickname, in case application needs it. It is also guaranteed in Toolkit 1.1 that the nickname inside SilcCientEntry is always valid nickname.
     
    The SilcChannelEntry now includes new channel_pubkeys list, which includes the channel public keys if they have added to the channel. This information was not present in Toolkit 1.0.
     
    The SilcServerEntry now includes new field public_key, which is the server's public key if we have resolved it. This information was not present in Toolkit 1.0.
     
    In Toolkit 1.1 it is now easier to search and resolve entries. As a new feature it now also possible to search and resolve server entries from the SILC network. See the API documentation for all the details on searching entries from the client library cache and from the SILC network.

    Sending/Receiving messages

    Sending messages has not much changed from the Toolkit 1.1. In Toolkit 1.1 the message may now safely be sent in from multiple threads for same client or for same channel. In case the message are digitally signed the hash function used in the signature computation must now be given as argument. If this is done in multiple threads, each thread must use different SilcHash context because SilcHash does not support multiple threads. Simply allocate new SilcHash for each thread where you send digitally signed messages.
     
    Receiving message is same as in Toolkit 1.0. You receive private_message or channel_message client operation. It is always guaranteed that even in multithreaded application the messages are received in one thread. You need concurrency control in your application only if you access shared data in your client operation callbacks.
     
    In Toolkit 1.1 as a new feature it is also possible to wait for incoming private messages in a thread. New function silc_client_private_message_wait can be used to block the calling process or thread until private message for the specified client is received.

    Calling and sending commands

    Just like in Toolkit 1.0 in Toolkit 1.1 you can call command implemented inside the client library with silc_client_command_call. The command_reply client operation will be called once the command reply has arrived.
     
    As a major change in semantics of sending commands with the silc_client_command_send function is the way the command reply is handled in Toolkit 1.1. In the new Toolkit the command_reply client operation will not be anymore called for commands that has been sent with silc_client_command_send. The command_reply client operation is called only when silc_client_command_call function is used. With silc_client_command_send you can give the command reply callback, SilcClientCommandReply, as argument, and it will be called for each command reply that is received from the server.
     
    Just likein 1.0 in 1.1 it is also possible to attach to pending commands by using silc_client_command_pending. As a difference to 1.0 the command identifier is not anymore available to application from the SilcClientConnection context. Instead the silc_client_command_call and silc_client_command_send return the command identifier, and application needs to save it in order to be able to attach to it at later time. However, this feature is not expected to be very important for application programmers as the new silc_client_command_send already includes the command reply callback.
     
    Comparison between 1.0 and 1.1
    
    Toolkit 1.0:
    /* Send ping command.  The reply will be in the attached command reply
       and in the command_reply client operation. */
    silc_client_command_send(client, conn, SILC_COMMAND_PING, ++conn->cmd_ident,
    			 1, 1, server_id, server_id_len);
    silc_client_command_pending(conn, SILC_COMMAND_PING, conn->cmd_ident,
                                ping_command_reply, context);
    
    Toolkit 1.1:
    /* Send ping command.  The reply will be ping_command_reply function. */
    silc_client_command_send(client, conn, SILC_COMMAND_PING,
                             ping_command_reply, context,
    			 1, 1, server_id, server_id_len);
    
    

    Notify arguments

    In Toolkit 1.1 the following notify arguments have had changes. See the Notify Arguments for details. You should go through your application and change the handling of the following notify messages for Toolkit 1.1.
     
    SILC_NOTIFY_TYPE_NICK_CHANGE
    SILC_NOTIFY_TYPE_CMODE_CHANGE
    SILC_NOTIFY_TYPE_SIGNOFF
    SILC_NOTIFY_TYPE_SERVER_SIGNOFF

     
    The basic changes in notify arguments from Toolkit 1.0 is that the Toolkit 1.1 parses various lists and other raw data for the application as opposed to sending them in the raw format. This makes programming easier.

    Command reply arguments

    In Toolkit 1.1 the following command reply arguments have had changes. See the Command Reply Arguments for details. You should go through your application and change the handling of the following command replies for Toolkit 1.1.
     
    SILC_COMMAND_WHOIS
    SILC_COMMAND_INVITE
    SILC_COMMAND_STATS
    SILC_COMMAND_JOIN
    SILC_COMMAND_CMODE
    SILC_COMMAND_BAN
    SILC_COMMAND_DETACH
    SILC_COMMAND_USERS

     
    The basic changes in command reply arguments from Toolkit 1.0 is that the Toolkit 1.1 parses various lists and other raw data for the application as opposed to sending them in the raw format.. This makes programming easier.

    Other changes in client library

    There are many other smaller changes in Toolkit 1.1 that require you to change your application when porting from Toolkit 1.0. We are not listing all of them here but briefly mention some API changes.
     
    Listing channel private keys now return SilcDList instead of an array.
     
    The key agreement API has changed a little bit and is now more cleaner and supports all the features that are needed in full featured key agreement. The silc_client_peform_key_agreement_fd has been replaced by silc_client_perform_key_agreement_stream.
     
    The private message key API has slight changes also. It is no longer necessary for the caller to specify whether the private message key is for responder or initiator use.
     
    The file transfer API has changed a little bit and is now more cleaner and supports all the features that are needed in full featured file transfer. It is now easier to send files when you are behind NAT when you can specifically define the IPs that are used in both file sending and receiving.
     
    As a new function silc_client_nickname_format can now be used to format the nickname of a client entry. The client library automatically formats the nicknames but in some cases application might like to change the nickname of a certain client entry.
     

    Utility library, lib/silcutil/

    The Utility library (runtime library) has had several changes and has several new interfaces. Some interfaces has also been removed or moved to some other library. Removed interfaces rae: silcprotocol.h and silcsockconn.h. Moved interfaces are: silcapputil.h and silcvcard.h.

    SILC Async Operation Interface

    A new asynchronous operation API (silcasync.h) has been added. It can be used to control asynchronous operations, like to cancel them. Many asynchronous routines in SILC Toolkit now return SilcAsyncOperation context so that the operation can be controlled by the caller. It especiallly provides a generic way to cancel asynchronous operations which can be difficult.

    SILC Atomic Operations Interface

    A new atomic operations API (silcatomic.h) has been added. It provides routines to perform various operations on integeres and pointers atomically.

    SILC Data Stack Interface

    A new data stack (memory pool system) API (silcstack.h) has been added. It provides a fast memory allocation system. Many routines in the SILC Toolkit are SilcStack aware thus enabling them to use the SilcStack as their source for memory allocation. All routines that are SilcStack aware automatically revert back to normal memory allocation if SilcStack is not given as argument. See silcstack.h for list of utility routines that support SilcStack by default.

    SILC Condition Variable Interface

    A new condition variable API (silccond.h) has been added. It provides condition variables for multithreaded applications.

    SILC Stream Interface

    A new abstract stream API (silcstream.h) has been added. The SilcStream provides an abstract way of representing different kinds of streams. The API provides functions that can be used to read, write, control and destroy streams. The API is not used to create streams but separate interfaces exist for streams that use the SilcStream abstraction. For example, socket stream and file descriptor stream exist.

    SILC FD Stream Interface

    A new file descriptor stream API (silcfdstream.h) has been added. It provides a blocking and non-blocking file descriptor stream through the SilcStream abstraction.

    SILC Socket Stream Interface

    A new socket stream API (silcsocketstream.h) has been added. It provides a blocking and non-blocking socket stream through the SilcStream abstraction.

    SILC FSM Interface

    A new Finite State Machine API (silcfsm.h) has been added. It provides an FSM that can be used to implement all kinds of machines and protocols. The machine also supports threads, and threads that are actually executed in real system threads. The SILC FSM API also supports asynchronous events.

    SILC Time Interface

    A new SILC Time API (silctime.h) has been added. It provides utility functions to retrieve and represent time in different ways. It supports Universal and Generalized time string creation and parsing and adds a new SilcTime structure to represent time.

    SILC Snprintf Interface

    A new snprintf API (silcsnprintf.h) has been added. It provides snprintf and other string formatting routines.

    SILC Mutex Interface changes

    The SILC Mutex API (silcmutex.h) has several changes. A support for read/write locks has been added (SilcRwLock). Also silc_mutex_assert_locked function is added.

    SILC Network Interface changes

    The SILC Network API (silcnet.h) has several changes. The API is almost entirely rewritten and most of the old functions have been removed. The API has now both TCP and UDP support, and as previously supports IPv4 and IPv6. New functions are silc_net_tcp_create_listener, silc_net_listener_get_port, silc_net_close_listener, silc_net_tcp_connnect, silc_net_udp_connect, silc_net_udp_receive, silc_net_udp_send.

    SILC Scheduler Interface changes

    The SILC Schedule API (silcschedule.h) has several changes. The scheduler has been entirely rewritten but most of the API remains the same. The SILC_TASK_GENERIC and SILC_TASK_CALLBACK_GLOCAL have been removed. The way signal are dispatched has been changed. The SILC Schedule is now able to itself dispatch all signals. New functions are silc_schedule_task_add_fd, silc_schedule_task_add_timeout, silc_schedule_task_add_signal, silc_task_del_by_all, silc_schedule_get_fd_events. The functions silc_schedule_signal_register, silc_schedule_signal_unregister and silc_schedule_signal_call have been removed.

    SILC Types Interface changes

    The SILC Type API (silctypes.h) has several changes. The bool type is replaced with SilcBool. Sockets are now represented by SilcSocket.

    SILC String util Interface changes

    The SILC string utility API (silcstrutil.h) has changes. The PEM encoding and decoding routines has been renamed, silc_base64_encode, silc_base64_encode_file and silc_base64_decode. The silc_mime_parse has been removed. A new silc_string_split function has been added.

    SILC Utility Interface changes

    SILC File Util Interface changes

    The SILC file utility API (silcfileutil.h) has changes. A new function silc_file_set_nonblock has been added.

    SILC List and Dynamic List Interface changes

    The SILC List (silclist.h) and SILC Dynamic List (silcdlist.h) APIs have changes. New functions silc_list_insert and silc_dlist_insert have been added.

    SILC Buffer Interface changes

    The SILC Buffer API (silcbuffer.h) has several changes. The SilcBuffer structure no longer contain the buffer length and true length fields but silc_buffer_len() and silc_buffr_truelen() macros are available instead. Also silc_buffer_data(), silc_buffer_datalen(), silc_buffer_purge(), silc_buffer_reset(), silc_buffer_start(), silc_buffer_end() and silc_buffer_enlarge() has been added. The API also supports SilcStack.

    SILC Buffer Formatting Interface changes

    The SILC Buffer Formatting API (silcbuffmt.h) has several changes. The silc_buffer_format now automatically allocates memory to the destination buffer if it does not have space. Also new the following new formatters have been added: SILC_STR_DATA (replaces SILC_STR_UI_XNSTRING), SILC_STR_BUFFER, SILC_STR_FUNC, SILC_STF_OFFSET and SILC_STR_ADVANCE. The API also supports SilcStack.

    SILC Memory Interface changes

    The memory allocation API (silcmemory.h) has several changes. It supports now SilcStack as memory source. Also all memory allocation routines can now fail and return NULL as opposed to fatally failing when memory allocation fails.
     

    SILC Application Utility library, lib/silcapputil/

    A new SILC Application Utility library has been added. It provides various application specific utility libraries that are not part of the runtime library (lib/silcutil/). The interfaces in the Application utility library were in other libraries in Toolkit 1.0 and the library does not contain any entirely new interfaces.

    SILC Application Utility Interface

    The silcapputil.h contains various application utility functions. It existed in Toolkit 1.0 but some of the APIs has been changed. The silc_create_key_pair, silc_load_key_pair and silc_show_public_key APIs has changed. A new silc_show_public_key_file has been added. Functions silc_identifier_check, silc_identifier_verify, silc_channel_name_check, silc_channel_name_verify, silc_get_mode_list silc_get_status_message, silc_get_packet_name, silc_get_command_name, silc_parse_version_string, silc_version_to_num, silc_client_chmode, silc_client_chumode, silc_client_chumode_char and silc_id_render has been moved from other libraries into this interface in Toolkit 1.1.

    SILC ID Cache Interface

    The ID Cache interface (silcidcache.h) has been moved from lib/silccore into lib/silcapputil/.
     

    SILC Key Repository library, lib/silcskr/

    A new SILC Key Repository library has been added. The library provides a SILC Key Repository API (silcskr.h) which provides a repository for storing and retrieving public keys.
     

    SILC VCard library, lib/silcvcard/

    A new SILC VCard library has been added. The SILC VCard API has been moved from utility library to own library in lib/silcvcard/.
     

    SILC HTTP library, lib/silchttp/

    A new SILC HTTP library has been added. The library includes SILC HTTP Server Interface and SILC HTTP PHP Translator Interface.

    SILC HTTP Server Interface

    The SILC HTTP Server API (silchttpservder.h) provides a simple HTTP server implementation for applications that want to integrate a small HTTP server.

    SILC HTTP PHP Translator Interface

    The SILC HTTP PHP Translator API (silchttpphp.h) provides PHP translates PHP code into HTML. It can be used to serve PHP pages in HTTP server.
     

    SILC ASN.1 library, lib/silcasn1/

    A new Abstract Syntax Notation One (ASN.1) library has been added. The library provides SILC ASN.1 encoder and decoder interface and SILC BER encoder and decoder interface.

    SILC ASN.1 Interface

    The SILC ASN.1 API (silcasn1.h) provides ASN.1 encoder and decoder. The interface provides efficient encoder and decoder and is support SilcStack as memory source. The interface is simple and it supports almost all ASN.1 features.

    SILC BER Interface

    The SILC BER API (silcber.h) provides BER/DER encoder and decoder. It is integral part of the ASN.1 library and the ASN.1 encoder and decoder.








  • Copyright © 2001 - 2007 SILC Project
    SILC Project Website
    SILC Toolkit Reference Manual
    Index