Find the answer to your Linux question:
Results 1 to 2 of 2
Hi all, I note that there is performance difference on 2 of our testing Dell PowerEdge 1950 (8GB RAM each, SAS 6/iR controller, kernel 2.6.18-92.1.6.el5). The only difference is disk: ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Mar 2007
    Posts
    6

    Pls help: slow fsync?


    Hi all,

    I note that there is performance difference on 2 of our testing Dell
    PowerEdge 1950 (8GB RAM each, SAS 6/iR controller, kernel
    2.6.18-92.1.6.el5). The only difference is disk:
    machine A: 2 x 146GB (Seagate ST3146855SS, 15000rpm, RAID 1)
    machine B: 2 x 500GB (Seagate ST3500630NS, 7200rpm, RAID 1)


    Both systems has drive write back cache enabled as shown in dmesg and LSIUTIL. Some IO performance test tools show results as expected (machine A is a bit faster than machine B due to faster hard disks):

    - dd if=/dev/zero of=bigfile count=10240 bs=1M; sync
    - iozone
    - bonnie++

    However, when I run load test with the code attached below that I
    borrowed from the net, performance of machine B is almost 4 times of that of machine A:

    ----------------------------- cut here --------------------------------
    #include <stdio.h>
    #include <unistd.h>
    #include <sys/types.h>
    #include <sys/stat.h>
    #include <fcntl.h>


    #define NUM_BLKS 200
    #define BLK_SIZE 4096 /* use either 20 or 4096 */


    char buff[BLK_SIZE * NUM_BLKS];


    int main(int argc, char * argv[])
    {
    int fd, k;


    fd = open("tst.txt", O_WRONLY | O_CREAT, 0644);
    for (k = 0; k < NUM_BLKS; ++k) {
    write(fd, buff + (k * BLK_SIZE), BLK_SIZE);
    fdatasync(fd);
    }
    close(fd);
    return 0;



    }

    ----------------------------- cut here --------------------------------

    I've no idea why and thus run a simple systemap script against this program:


    probe kernel.function("*@fs/buffer.c").return {
    printf ("%s <- %s(%d)\n", thread_indent(-1), probefunc(),
    gettimeofday_ms())
    }

    Found that on the machine A, number of calls in buffer.c is much greater (almost 4 times) than machine B:

    alloc_buffer_head 130 556
    alloc_page_buffers 130 556
    __block_commit_write 4787 19489
    __block_prepare_write 4792 19854
    block_prepare_write 9583 39709
    block_sync_page 4523 18731
    __block_write_full_page 4783 19834
    block_write_full_page 9565 39660
    __bread 258 1083
    __brelse 666 2959
    create_empty_buffers 130 554
    __do_fsync 4663 19308
    do_fsync 9327 38616
    __find_get_block 656 2813
    __find_get_block_slow 138 697
    generic_commit_write 4787 19842
    __getblk 656 2804
    init_page_buffers 0 2
    inode_has_buffers 0 2
    invalidate_inode_buffers 0 2
    ll_rw_block 0 6
    mark_buffer_dirty 4789 19852
    recalc_bh_state 130 556
    set_bh_page 130 556
    submit_bh 4786 19838
    sys_fdatasync 4663 19303
    unlock_buffer 785 3356
    unmap_underlying_metadata 129 552


    Seems the performance difference is caused by the number of calls
    requested - looks like a smaller write buffer cache is used on one machine
    and thus needs more fsync to flush data to disk. However, I got stuck
    here and have no idea about the actual reason behind. Would anyone
    pls help? Sorry for the newbie question.

    Thanks a lot.
    /ST Wong

  2. #2
    Just Joined!
    Join Date
    Mar 2007
    Posts
    6
    Some more findings on this issue:

    I run performance test tool of BerkeleyDB on both systems. Performance differs a lot when running the 'sync-on-commit' test that calls fdatasync. Using top and ps, found that WCHAN of one system always shows log_commit_wait. While all filesystems are ext3 mounted with default option (data=ordered), I wonder why fdatasync doesn't skip metadata update while both systems run the same kernel (2.6.18-92.1.6.el5)...

    Would anyone pls help? Thanks a lot.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •