Random Scribbling

Tools

2010-09-28T02:48:00.000-07:00

Here is the list of the tools that I use on a daily basis.

a) awesome Window Manager http://awesome.naquadah.org/

b) GNU Emacs editor http://www.gnu.org/software/emacs/

c) notmuch Mail client http://notmuchmail.org/

d) iceweasel/google chrom. I would completely switch to google chrome if i can find a replacement to iceweasel scrapbook addon

e) GNU Screen http://www.gnu.org/software/screen/

f) Gnome-do http://do.davebsd.com/

g) hamster-applet http://projecthamster.wordpress.com/ (planning to switch to emacs org-mode)

h) gtg http://live.gnome.org/gtg (planning to switch to emacs org-mode)

i) GNU global http://www.gnu.org/software/global/

j) Git version control http://git-scm.com/

k) stgit http://www.procode.org/stgit/

debug-btrfs (btrfs file system debugger)

2010-02-04T10:07:00.000-08:00

One of the easiest way to learn any specific file system layout is to write tools that can read the layout in user space. I started with ext4migrate for ext4. Now I am writing debug-btrfs for btrfs file system. Hope this will be useful for others too. Btrfs is a new copy on write filesystem for Linux and it makes extensive use of btrees. Luckily Chris Mason had done all the hard work of writing the library for getting different file system objects. So writing tools is just a matter of figuring out what you should look for in the tree.

Rich-acl patches posted

2010-02-04T08:55:00.000-08:00

You can find the patches here .

Patches implements a new acl format for linux so that we can have better acl interoperability with CIFS and NFSv4 acl.

channel select plugin for xchat

2010-01-14T23:33:00.000-08:00

This works with xchat. I was not able to get the /gui focus command to work on xchat-gnome.

__module_name__ = "chan"
__module_version__ = "0.1"
__module_description__ = "Channel select plugin"

import xchat

def chan_select(word, word_eol, userdata):
    if len(word) < 2:
        print "Second argument must be the channel name"
        return xchat.EAT_ALL
    else:
        context = xchat.find_context(channel=word[1])
        if context:
            context.command("gui focus")
        else:
            print "Channel not found " + word[1]

    return xchat.EAT_ALL

xchat.hook_command("chan", chan_select, help="/chan channel-name")
print "Channel select plugin loaded"

Update: Found this https://bugzilla.gnome.org/show_bug.cgi?id=600548 . So should work with the next release of xchat-gnome

linux kernel jbd2 notes

2009-11-13T00:27:00.000-08:00

Sharing some of the notes taken by me while looking at fs/jbd2 code earlier.

Journal commit
==============

1) We start with the running transaction. (journal->j_running_transaction)

2) We change the transaction state to T_LOCKED so that nobody gets a handle
   to this transaction when we are committing the transaction.

    When we do a jbd2_journal_start we look at the transaction t_state
 and if found T_LOCKED we do wait on j_wait_transaction_locked wait
  queue.

3) If somebody already have got a handle on this transaction wait for that
   transaction to do a journal_stop.  (j_wait_updates wait queue).
  
 jbd2_journal_stop does a wakeup on j_wait_updates.

4) Discard the contents of t_reserved_list. FIXME Not sure how we
   can have something in the reserved list. We generally add items to reserved
   list when we are trying to get write access to meta data buffers. ie, via
   jbd2_journal_get_write_access. But since we are preventing a journal_start
   above not sure how we can something on reserved list. The code comment mention
   truncate. 

5) Now switch the revoke table. That means any new revoke action will add to the
   new table. The old table will be written to the journal and we want to make
   sure nobody update the old table.

6) Change the journal state to T_FLUSH

7) mark the transaction as committing  transaction 
    journal->j_committing_transaction = current_transaction

8) Mark the running transaction as NULL 
    journal->j_running_transaction = NULL;
 This would create a new transaction during the next journal_start. So at any
 time we can have one running transaction and one committing  transaction.

9) Wake up anybody waiting on the j_wait_transaction_locked wait queue. (See step 2).

10) Write the dependent data blocks of this transaction.
    Dependent data blocks are added during file system write. For example ext4_ordered_write_end
    does ext4_jbd2_file_inode which will file the inode in the transactions inode list.This implies
    in order to commit this transaction we need to ensure that all dirty pages of the inode need
    to be written to disk. That ensure that we don't have meta data on disk that point to blocks
    carrying old data.

11) Write revoke records. This will allocate a journal block of JBD2_REVOKE_BLOCK type and add
    revoke records in the journal. After writing the revoke block we add the revoke block details
    to BJ_LogCtl list. The revoke record are used during replay so that we don't replay recorded
    journal changes for specific blocks. These records are created when file system do free a blocks.
    For example ext4 calls ext4_forget on the meta data blocks freed and that does jbd2_journal_revoke.
    

12) Change the journal state to T_COMMIT

13) Loop through the meta data blocks added to the journal. We look at transaction->t_buffers. That is
    the BJ_Metadata list. The meta data blocks get added to the list after the file system have modified
    the meta data. For example ext4 calls ext4_handle_dirty_metadata after updating meta data values.
    It does jbd2_journal_dirty_metadata  which marks the journal_head->b_modified = 1  and then adds it
    to the current transaction's BJ_Metadata list, thereby removing it from the BJ_Reserved list.

14) Allocate a descriptor block (block type JBD2_DESCRIPTOR_BLOCK) if we don't have one. A descriptor block
    contain a journal_header followed by journal block tag. There can be as many journal block tag that
    can fit in a journal descriptor block. Each journal block tag will carry the block number of the file
    system that is being put into the journal. So What we have is something below.

    | journal_header_t | journal_block_tag_t 1 | journal_block_tag_t 2|......| metata data block 1 | meta data block 2|

    journal_block_tag 1 will contain information which file system block number maps to meta data block 1.

    All the descriptor blocks are added to BJ_LogCtl list of the transaction. 

15) Get a journal block for writing meta data. ( See step 14)

16) Make a copy of the meta data content into the journal block. We use frozen data if present.
    Frozen data is used when we try to modify the meta data which is already part of a committing  transaction.
    For ex: ext4_get_write_access we check whether the buffer_head already part of committing transaction.
    In detail, during do_get_write_access if we find the journal_head mapping buffer_head to be in BJ_Shadow
    list (we put buffer_heads in BJ_Shadow list when we having started the I/O to disk). We wait for the
    IO to be over. We do that by waiting  for the buffer_head flag to set BH_Unshadow. If we are not on
    BJ_Shadow list and also not on BJ_Forget list. (We are on BJ_Forget list when we have finished IO
    and are going to be added to checkpoint list) we make a copy of the existing buffer_head data to forzen
    data. This make sure what we write to the disk doesn't get modified beneath us. We also update the 
    journal_head next transaction to be the transaction in which we are making the update. This make sure
    once we are done with the I/O instead of removing the meta data block from the journal we add it back
    to the next transaction.

       We also need to make sure we update the journal metata data content in such a way that we don't
       have data matching JBD2_MAGIC_NUMBER which would make it consider as a journal internal block.So
       we do necessary data modification and capture that information in journal block tag.

17) Add the meta data buffer to BJ_Shadow list

18) Add the journal block to the BJ_IO list 

19) Start writing the journal block 

20) Wait for the dependent data block to finish writing. (See step 10)

21) Wait for the t_io_buf_list  IO to finish. These are buffers in BJ_IO list.
    they are added in step 18

22) Remove the journal blocks from the BJ_IO list

23) Remove the mapping meta data blocks from the BJ_Shadow list (t_shadow_list)

24) Add the metadata to BJ_Forget list for tracking checkpoint

25) Wake up anybody waiting on the shadow list (see step 16) 
      wake_up_bit(&bh->b_state, BH_Unshadow);

26) Wait for t_log_list (BJ_logCtl) to finish IO (see step 11 and 14)

27) Now that we know that we have written everything to the journal write the commit record.
    commit record have block type JBD2_COMMIT_BLOCK.

28) Wait on commit record to finish IO.

29) Loop through the t_forget_list (BJ_Forget) and if the journal head is found dirty add to
    checkpoint list of transaction.
     
 We add meta data blocks which mapping journal blocks are already written to
 the journal to the BJ_Forget list.

30) If the meta data buffer is part of next transaction (see step 16) refile the buffer in
    either BJ_Reserved or BJ_Meta depending on whether the data is already modified or not
    (step 13)

31) If not mark the meta data block as dirty thereby exposing the data to VM (__jbd2_journal_unfile_buffer)

32) If the transaction have checkpoint list add it to the journal checkpoint transaction list.

33) As we request for space from the journal and if we find no space we write the blocks in the
    checkpoint list thereby freeing the journal space for reuse.


Journal Recovery
==============

1) Starts with commit id in journal superblock->s_sequence and with start block in journal
   superblock->s_start.

2) Read the start blocks which is the logical block number of the journal inode

3) Look at the header and find out what block type.

4) if block type is descriptor block look at the journal block tag and find the file system
   block number. (see step 14)

5) replay the journal block which is immediately following the descriptor block. Consult the
   revoke records to check whether the block should be replayed or not.

6) Do this till commit record is hit

   The layout is:
[revoke record block][ descriptor block ][data][data][descriptor block][data]..[commit]

Second attempt at POSIX NFSv4 acl

2009-09-02T09:48:00.000-07:00

Posted the RFC patch that did POSIX ACL NFSv4 client side support using sideband protocol.
http://linux-nfs.org/pipermail/nfsv4/2009-September/011142.html
previous one which mapped the v4 acl to posix acl on the client side can be found
http://linux-nfs.org/pipermail/nfsv4/2009-July/010768.html

onashamsakal

2009-09-02T08:21:00.000-07:00

Happy onam

emacs auto-complete and ispell integration

2009-07-28T12:05:00.000-07:00

(require 'auto-complete)
(require 'ispell)

(defvar ac-ispell-modes
  '(text-mode))

(defun ac-ispell-candidate ()
(if (memq major-mode ac-ispell-modes)
(let ((word (ispell-get-word nil "\\*")))
     (setq word (car word))
     (lookup-words (concat word "*") ispell-complete-word-dict))))

(defvar ac-source-ispell
  '((candidates . ac-ispell-candidate)
    (requires . 3))
  "Source for ispell.")

(provide 'auto-complete-ispell)

emacs frame made easy in no-window mode

2009-06-30T00:17:00.000-07:00

Now that we have separate tag stack for each frame, I wanted to have the something similar to vim tab support. Emacs wiki is an excellent resource for finding new emacs extensions. For tab support i found

o tabbar mode
o elscreen

Of the above what I wanted was elscreen. I liked elscreen because of its similarity to screen. But getting separate tag stack for multiple elscreen window turned out to be complex.

The solution was to see how easy to get the emacs frame support do what i wanted. Emacs no-window-mode already allowed creating multiple frames. So what remained is to have a sane key binding that allowed creating new frames and moving between multiple frames easily. Here is what i ended up writing.



(defvar frame-browse-mode-map (make-sparse-keymap)
  "Keymap used in frame-browse mode.")

(defun select-frame-1 ()
(interactive)
(select-frame-by-name "F1"))

(defun select-frame-2 ()
(interactive)
(select-frame-by-name "F2"))

(defun select-frame-3 ()
(interactive)
(select-frame-by-name "F3"))

(defun select-frame-4 ()
(interactive)
(select-frame-by-name "F4"))

(defun select-next-frame ()
(interactive)
(other-frame -1))

(defun select-previous-frame ()
(interactive)
(other-frame 1))


(define-key frame-browse-mode-map "c" 'make-frame-command)
(define-key frame-browse-mode-map "1" 'select-frame-1)
(define-key frame-browse-mode-map "2" 'select-frame-2)
(define-key frame-browse-mode-map "3" 'select-frame-3)
(define-key frame-browse-mode-map "4" 'select-frame-4)
(define-key frame-browse-mode-map "n" 'select-next-frame)
(define-key frame-browse-mode-map "p" 'select-previous-frame)

(global-set-key "\C-t" frame-browse-mode-map)

Linux kernel and gtags changes

2009-06-23T11:45:00.000-07:00

Linux kernel have these complex syscall indirection using macros like

SYSCALL_DEFINE3(open, const char __user *, filename, int, flags, int, mode) 
SYSCALL_DEFINE(fallocate)(int fd, int mode, loff_t offset, loff_t len)

Getting the source code tag system to understand them is going to be difficult. ctags support regex based substitution that help to translate the above definition to sys_open and sys_fallocate. To achieve the same in global i had to patch global.


diff --git a/gtags-parser/C.c b/gtags-parser/C.c
index c5653aa..635e8bd 100644
--- a/gtags-parser/C.c
+++ b/gtags-parser/C.c
@@ -143,6 +143,7 @@ C_family(const char *file, int type)
     } else if (level == 0 && !startmacro && !startsharp) {
      char arg1[MAXTOKEN], savetok[MAXTOKEN], *saveline;
      int savelineno = lineno;
+     char *linux_kernel;
 
      strlimcpy(savetok, token, sizeof(savetok));
      strbuf_reset(sb);
@@ -166,6 +167,12 @@ C_family(const char *file, int type)
      if (function_definition(target, arg1)) {
       if (!strcmp(savetok, "SCM_DEFINE") && *arg1)
        strlimcpy(savetok, arg1, sizeof(savetok));
+      linux_kernel = getenv("LINUX_KERNEL_SOURCE");
+      if (linux_kernel && !strcmp(linux_kernel, "yes") &&
+       !strncmp(savetok, "SYSCALL_DEFINE", 14 ) && *arg1) {
+       strcpy(savetok, "sys_");
+       strncat(savetok, arg1, sizeof(savetok) - 4);
+      }
       if (target == DEF)
        PUT(savetok, savelineno, saveline);
      } else {
@@ -611,7 +618,7 @@ function_definition(int target, char arg1[MAXTOKEN])
  if (c == EOF)
   return 0;
  brace_level = 0;
- while ((c = nexttoken(",;[](){}=", c_reserved_word)) != EOF) {
+ while ((c = nexttoken(";[](){}=", c_reserved_word)) != EOF) {
   switch (c) {
   case SHARP_IFDEF:
   case SHARP_IFNDEF:

global/gtags support for per frame tag stack

2009-06-23T00:31:00.001-07:00

The below patch should get you per frame tag stack.

diff --git a/gtags.el b/gtags.el
index bbe4e8a..1eb3f23 100644
--- a/gtags.el
+++ b/gtags.el
@@ -69,12 +69,6 @@
   :group 'gtags)
 
 ;; Variables
-(defvar gtags-current-buffer nil
-  "Current buffer.")
-(defvar gtags-buffer-stack nil
-  "Stack for tag browsing.")
-(defvar gtags-point-stack nil
-  "Stack for tag browsing.")
 (defvar gtags-history-list nil
   "Gtags history list.")
 (defconst gtags-symbol-regexp "[A-Za-z_][A-Za-z_0-9]*"
@@ -165,22 +159,34 @@
 
 ;; push current context to stack
 (defun gtags-push-context ()
-  (setq gtags-buffer-stack (cons (current-buffer) gtags-buffer-stack))
-  (setq gtags-point-stack (cons (point) gtags-point-stack)))
+  (let (cur-frame (selected-frame))
+  (setq gtags-buffer-stack (cons (current-buffer) (frame-parameter cur-frame 'buffer-stack)))
+  (modify-frame-parameters cur-frame `((buffer-stack . ,gtags-buffer-stack)))
+  (setq gtags-point-stack (cons (point) (frame-parameter cur-frame 'point-stack)))
+  (modify-frame-parameters cur-frame `((point-stack . ,gtags-point-stack)))))
 
 ;; pop context from stack
 (defun gtags-pop-context ()
-  (if (not gtags-buffer-stack) nil
-    (let (buffer point)
-      (setq buffer (car gtags-buffer-stack))
-      (setq gtags-buffer-stack (cdr gtags-buffer-stack))
-      (setq point (car gtags-point-stack))
-      (setq gtags-point-stack (cdr gtags-point-stack))
+  (if (not (frame-parameter (selected-frame) 'buffer-stack)) nil
+    (let (buffer point (cur-frame (selected-frame)))
+      (setq buffer (car (frame-parameter cur-frame 'buffer-stack)))
+      (setq gtags-buffer-stack (cdr (frame-parameter cur-frame 'buffer-stack)))
+      (modify-frame-parameters cur-frame `((buffer-stack . ,gtags-buffer-stack)))
+      (setq point (car (frame-parameter cur-frame 'point-stack)))
+      (setq gtags-point-stack (cdr (frame-parameter cur-frame 'point-stack)))
+      (modify-frame-parameters cur-frame `((point-stack . ,gtags-point-stack)))
       (list buffer point))))
 
+(defun gtags-get-current-buffer ()
+  (frame-parameter (selected-frame) 'g-current-buffer))
+
+(defun gtags-set-current-buffer (buffer)
+  (modify-frame-parameters (selected-frame) `((g-current-buffer . ,buffer))))
+
+
 ;; if the buffer exist in the stack
 (defun gtags-exist-in-stack (buffer)
-  (memq buffer gtags-buffer-stack))
+  (memq buffer (frame-parameter (selected-frame) 'buffer-stack)))
 
 ;; get current line number
 (defun gtags-current-lineno ()
@@ -396,9 +402,9 @@
   "Move to previous point on the stack."
   (interactive)
   (let (delete context buffer)
-    (if (and (not (equal gtags-current-buffer nil))
-             (not (equal gtags-current-buffer (current-buffer))))
-         (switch-to-buffer gtags-current-buffer)
+    (if (and (not (equal (gtags-get-current-buffer) nil))
+             (not (equal (gtags-get-current-buffer) (current-buffer))))
+         (switch-to-buffer (gtags-get-current-buffer))
       (if (not (gtags-exist-in-stack (current-buffer)))
    (setq delete t))
       (setq context (gtags-pop-context))
@@ -407,7 +413,7 @@
         (if delete
      (kill-buffer (current-buffer)))
         (switch-to-buffer (nth 0 context))
-        (setq gtags-current-buffer (current-buffer))
+        (gtags-set-current-buffer (current-buffer))
         (goto-char (nth 1 context))))))
 
 ;;
@@ -520,7 +526,7 @@
         ;; move to the context
         (if gtags-read-only (find-file-read-only file) (find-file file))
         (if delete (kill-buffer prev-buffer)))
-      (setq gtags-current-buffer (current-buffer))
+      (gtags-set-current-buffer (current-buffer))
       (goto-line line)
       (gtags-mode 1))))
 
@@ -592,7 +598,7 @@ Turning on Gtags-Select mode calls the value of the variable
  truncate-lines t
         major-mode 'gtags-select-mode
         mode-name "Gtags-Select")
-  (setq gtags-current-buffer (current-buffer))
+  (gtags-set-current-buffer (current-buffer))
   (goto-char (point-min))
   (message "[GTAGS SELECT MODE] %d lines" (count-lines (point-min) (point-max)))
   (run-hooks 'gtags-select-mode-hook))

vim like -t support in emacs

2009-06-21T09:59:00.000-07:00

(defun arg-find-tag(dummy)
  (gtags-goto-tag (car command-line-args-left) "")
  (setq command-line-args-left (cdr command-line-args-left)))

(add-to-list 'command-switch-alist '( "--tag". arg-find-tag))

That would get you emacs --tag

She is keeping me really busy

2009-06-21T09:50:00.000-07:00

Primary reason you haven't seen much blog update from me for a long time was because of her. She is sooooooo cute that I always want to be around her. Arya is now 6 months old.

Switching from vim to emacs

2009-06-21T09:24:00.000-07:00

I decided to try emacs as the editor for all development work primarily because i was not able to find a proper source code taging system that worked with vim properly. ctags supported tag completion but lacked function reference support. Whereas cscope lacked tag completion. What motivated me to switch to emacs was the global source code tag system. vim editor's support for global is partial. vim editor needs source code changes to add a new tag file format. Another motivating factor was the ability to debug the code in the editor. That helps to use the source tag system to read code while you are in a debug session.

Now about the good and bad part of the switch. emacs key binding is really really painful. Literally, it hurts your arm if you keep using the default keybinding. I was also comfortable with vim like key binding where i had the ESC mode to give editor command and insert mode to edit. The solution was to use viper. Good part, I really like the integration with global. Being able to search with normal editor commands in the search result window is really nice. Also I now have tag completion and function reference support. (But i had to patch global to get some of the required features. Will do another blog post on that). Integration with gdb is really great. In the graphics mode i can have multiple windows showing different debugger details like register dump, stack frame etc.

Another thing that came in as a bonus is ability to extend editor features with a proper programming language. Though that was not a motivating factor for the switch, i now really like the fact that i can extend emacs using elisp.

So all in all i am happy with the switch.

Tuz in Blue

2009-04-07T07:05:00.000-07:00

Ext4 OLS paper

2008-07-26T08:19:00.000-07:00

OLS 2008 Proceedings can be found here . Ext4 paper is here

orkut account deleted

2008-05-19T12:47:00.000-07:00

I rarely used it and I guess orkut did some thing really wrong recently. Lot of spam scraps get posted to different people accounts. I got notified by many of my friends. At this point I am tempted to believe that it is an orkut bug since I also received similar spam scraps from other friends. My first attempt was to get the scraps cleaned. But then I noticed that the quiet a lot of junk communities got added to my account. So I decided to quit. Any way orkut is one of the google services I rarely used

Hunting the file system corruption bug.

2008-04-09T10:38:00.000-07:00

As I was starting to enjoy the OLS paper my restless brain decided to run the latest and greatest ext4 patchqueue on all architectures ie, x86, x86_64 and powerpc.I was mostly expecting a full PASS status. But things don't always go that smooth right ?. The test failed, and that too only on powerpc. The failure happened with the new test case I added to run fsstress with falloc support.

With the failed test on one hand and the paper to finish on the other I was put in a confused state and found it difficult to decide what to work on. Should i start looking at the failure ?. That could possibly jeopardize the paper due to lack of time. So I avoided looking at the failure till evening. By then I was not able to stand the thought that one of the patch that I did for fallocate could have caused the failure and it should be looked at. So I ditched the paper for the day and decided to take a look at the problem. It was not easy.

fsstress have set of operations that gets picked in a random fashion for the test. So finding out which set of operations resulted in the file system corruption is a difficult task. First step involved collecting more data out of the file system. We have 1000000 operations done in the test case and the file system is 100% full. That got me more worried, because I had set of patches to handle fallocate and 100% file system full case. Now searching 1000000 operations to find out which one caused the corruption is not an easy task. So I decided to reduce the number of operations. First step is to make the file system partially full. Created a temp file of 8.5 GB and ran the test again with less number of operation. Luckily the problem got reproduced. Total number of operations reduced to 81136. With this I started to look at the detals. The inodes which are corrupted are listed by e2fsck. First step is to dump the inode and see what is wrong.

tst_extents:  stat <122402>
header: magic=af3 entries=512 max=1024 depth=0 generation=0
(Left 1)
(Left 0)
Inode: 122402   Type: regular    Mode:  0666   Flags: 0x80000   Generation: 447577215
User:     0   Group:     0   Size: 0
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 32
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x47fba327 -- Tue Apr  8 22:23:59 2008
atime: 0x47fba31d -- Tue Apr  8 22:23:49 2008
mtime: 0x47fba327 -- Tue Apr  8 22:23:59 2008
Size of extra inode fields: 28
BLOCKS:
(245-247):2419033-2419035, (248):2419038
TOTAL: 4

tst_extents:  inode <122402>
header: magic=af3 entries=512 max=1024 depth=0 generation=0
Loaded inode 122402
tst_extents:  root
(Left 1)
extent: lblk 245--247, len 3, pblk 2419033, flags: LEAF 
tst_extents:  nl
(Left 0)
extent: lblk 248--248, len 1, pblk 2419038, flags: LEAF 
tst_extents:  nl
nl: No 'next' extent 
tst_extents:

Well the header looks wrong, entries = 512 ?. That is pretty large. So did the tool got it wrong or is it really wrong in filesystem ? To find out i need to dump the blocks corresponding to inode.

tst_extents:  imap <122402>
Inode 122402 is part of block group 15
        located at block 491522, offset 0x0100
tst_extents:


So we need to dump 491522 block

dd if=/dev/sda7 of=/root/log bs=4096 skip=491522 count=1

hexdump /root/log and look at offset 0x0100

....
 0000100 b681 0000 0000 0000 1da3 fb47 27a3 fb47
 0000110 27a3 fb47 0000 0000 0000 0100 2000 0000
 0000120 0000 0800 0100 0000 0af3 0200 0400 0000
 0000130 0000 0000 f500 0000 0300 0000 59e9 2400
 0000140 f800 0000 0100 0000 5ee9 2400 0000 0000
 0000150 0000 0000 0000 0000 0000 0000 0000 0000
 0000160 0000 0000 7f7c ad1a 0000 0000 0000 0000
 0000170 0000 0000 0000 0000 0000 0000 0000 0000
 0000180 1c00 0000 7c58 5c7e 7c58 5c7e 48cf 909f
 0000190 1da3 fb47 48cf 909f 0000 0000 0000 0000
 00001a0 0000 0000 0000 0000 0000 0000 0000 0000

 ....

Now where is the extent header ... ?

0000120 0000 0800 0100 0000 0af3 0200 0400 0000

That have the right magic value f30a which is EXT4_EXT_MAGIC and the number of entries is 0002 and max entries 0004

So the tool is wrong. The tool is not byte swapping the values on powerpc.That also implied the extent format is not a corrupt. Rather we have a kernel bug which is adding new blocks but not updating i_size.

To find different operations happened on this file we need to map the inode to file. Well luckly I am still able to mount the file system.So i can do a find

find ffs4 -inum 122402 --print

That listed the filname and now i can look at the fsstress log to see what all operations happened on this file ? The operation listed

9/863: fallocate d1/d2d/d32/d8f/d97/d108/d9b/ffb [2041,75851] 28

So how can a failed fallocate cause two extents to be allocated ? That also with not uninit extents ? May be we are doing something wrong in fallocate under file system full case. So started reading the code multiple times again and again. I was worried that one of my patch handling the file system full case is doing some thing wrong.

By that time I had already hit 3.00 am and I decided to sleep on it rather than debug further.

Next day morning I check my mail and finds this.

"This may happens because of direct write. ...."

It tunrs out that directIO have some issues in i_size handling with ENOSPC http://lkml.org/lkml/2006/12/18/103

So i disable the directIO test from fsstress and run the tests again, and i get, "GOOD Completed Successfully"

What is still missing the above is the file doesn't have any directIO operation listed in fsstress logs. There are other files which had corrupt size that had directIO operation like the below.

18/733: fallocate d9/ddd/f86 [481025,18486] 0
18/735: dwrite d9/ddd/f86 [679936,36864] 0
18/764: dread d9/ddd/f86 [53248,8192] 0
18/910: dwrite d9/ddd/f86 [1454080,4096] 28
18/999: dwrite d9/ddd/f86 [1798144,61440] 28


Now to find out how those two extents end up, I started looking for directIO write at 245 block. Found these three

9/816: dwrite d1/d3c/d9b/ffb [1003520,32768] 28
9/826: dwrite d1/d2d/d32/d8f/d9c/ff8 [1003520,73728] 28
9/903: dwrite d1/dc/d14/fde [1003520,4096] 28


Searching for these files in the file.list showed that

$  grep ffb fs.list | grep d9b
122402 -rw-rw-rw-  1 root root 0 Apr  8 12:53 ffs4/p9/d1/d2d/d32/d8f/d97/d108/d9b/ffb
$

So did the file got unlinked

$ grep unlink fs.log | grep ffb  | grep d9b
$

That implies some of the data is missing. For the time being i left it there to be haunted again later.

That is how you fix file system corruption bugs :)

btrfs wiki

2008-03-31T11:58:00.000-07:00

Chris Mason announced btrfs wiki

Mutt vfolder

2008-03-28T10:00:00.000-07:00

I am finding little time to code the last two days since most of the time is spent on the difficult attempt to write the Ext4 paper. I am taking more breaks in between work because the entire exercise is not fun. Being not in a mood to look at new Ext4 stuff since getting started with new stuff would mean me not completing the paper, I decided to look at some of the TODO items i had.

One of the things i was missing in mutt was the ability to search all the folders and put the search results in a temporary folder aka vfolder. The Google search found this

While googling I also learnt a cool bash trick

func()
{
 read -p "Enter value:" $1
 echo ${!1}
 echo ${test}
}

func test

What is interesting above is, "test" is the variable name. Being able to decided the variable name runtime and store values in them seems really interesting.

Systemtap as a kernel learning tool

2008-03-24T08:20:00.000-07:00

Every now and then kernel newbies would face the situation wondering how exactly the job gets done by the kernel. Ex: Today while looking at why e2fsck is considering a deleted normal symlink as fast symlink i found this.

root@elm3b165:~# /usr/local/e2fsprogs/sbin/e2fsck  -f  /dev/sda5
e2fsck 1.40.8 (13-Mar-2008)
Pass 1: Checking inodes, blocks, and sizes
Fast symlink 12 has EXTENT_FL set.  Clear?

This is because after deleting the file we have
debugfs:  stat <12>
Inode: 12   Type: symlink    Mode:  0777   Flags: 0x80000   Generation:
25346603
User:     0   Group:     0   Size: 0
File ACL: 0    Directory ACL: 0
Links: 0   Blockcount: 0
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x47e39ea3 -- Fri Mar 21 11:40:19 2008
atime: 0x47e39e9e -- Fri Mar 21 11:40:14 2008
mtime: 0x47e39e9a -- Fri Mar 21 11:40:10 2008
dtime: 0x47e39ea3 -- Fri Mar 21 11:40:19 2008
Size of extra inode fields: 28
Fast_link_dest:

What this means is, while deleting the file it is getting truncated. So deletion is not just removing the entry from the directory. That makes sense because we will also have to free the blocks corresponding to the deleted inode.

Now we know that the truncation is done by ext4_truncate. But you don't find this function directly getting called from ext4_unlink. Now the hard way is to look around and find out how ext4_truncate get called from ext4_unlink or you can simply do

[aneesh@jataayu ~]$ stap  a8.tap
Called ext4_unlink
0xe0941282 : ext4_unlink+0x1/0x1dd [ext4dev]
0xc049139e : do_unlinkat+0x9c/0x100
0xc04914ff : sys_unlinkat+0x2d/0x2f
0xc0405d12 : syscall_call+0x7/0xb
Called ext4_truncate
0xe093d68f : ext4_truncate+0x1/0x53e [ext4dev]
0xc0499735 : generic_delete_inode+0x96/0x10e
0xc04997bf : generic_drop_inode+0x12/0x137
0xc0498ae6 : iput+0x63/0x66
0xc04913c2 : do_unlinkat+0xc0/0x100
0xc04914ff : sys_unlinkat+0x2d/0x2f
0xc0405d12 : syscall_call+0x7/0xb

There you go ext4_truncate get called from the iput().

The systemtap file is
[aneesh@jataayu ~]$ cat a8.tap
probe module("ext4dev").function("ext4_unlink")
{
 printf("Called %s\n", probefunc());
 print_backtrace();
}
probe module("ext4dev").function("ext4_truncate")
{
 printf("Called %s\n", probefunc());
 print_backtrace();
}
[aneesh@jataayu ~]$

4 years in REC (NITC)

2008-03-22T02:41:00.000-07:00

The Ext4 Paper got selected for OLS

Ext4 in Fedora 9 alpha

2008-02-11T05:48:00.000-08:00

http://fedoraproject.org/wiki/FedoraExt4

fs/jbd[2] explained

2008-02-03T22:57:00.001-08:00

Saturday lunch

2008-02-02T05:25:00.001-08:00

Mutli block allocator explained.

2007-12-21T22:47:00.000-08:00

http://article.gmane.org/gmane.comp.file-systems.ext4/4260

Multi block allocator and soft lockup

2007-12-21T22:44:00.000-08:00

3 days back when testing my block allocation optimization patches, i hit this strange soft lockup warning while discarding the multi block allocator prealloc space. I had never hit this hang before in my testing. Isolating the actual cause of the warning required quiet a lot of test boot kernel. My initial attempt was to see whether it matched the 2.6.24-rc5 soft lockup warning messages others are reporting. But that was mostly due to NO_HZ and the watchdog thread no being woken up. Later i noticed Ingo merged another soft lockup fix in the sched git tree. But that also didn't help. Then my attempt was to see if my optimization patches caused the warning. I was able to reproduce the hang without any changes on top of ext4 patch queue. Then i started looking for config options that would help debug this issue. I had to patch the bit_spinlock to throw a warning in case we got stuck there. But nothing turned. Then i noticed that CONFIG_PREEMPT was turned off. Enabling that option made the problem go away. This speeded up the debug quiet a lot, because we now know that we are getting stuck somewhere in the kernel in a tight loop without preemption disabled. So it is not a spin lock dead lock.

That made me read the block allocator again and after few test patches i came up with this. http://article.gmane.org/gmane.comp.file-systems.ext4/4360

FOSS.IN

2007-12-04T21:51:00.000-08:00

I am at FOSS.IN . I will be at the IBM stall most of the time. So if you want to discuss about ext4 please drop by

Mansar lake

2007-11-13T19:13:00.000-08:00

Mansar

2007-11-13T10:11:00.000-08:00

Google Code for Educators

2007-10-20T22:51:00.000-07:00

Here Tutorials Video Lectures Really good.

powerpc test_and_clear_bit and alignment requirement

2007-10-13T23:08:00.000-07:00

#define BITS_PER_LONG 64
#define BITOP_MASK(nr)        (1UL << ((nr) % BITS_PER_LONG))
#define BITOP_WORD(nr)        ((nr) / BITS_PER_LONG)

#define BITOP_LE_SWIZZLE    ((BITS_PER_LONG-1) & ~0x7)
#define __test_and_clear_le_bit(nr, addr) \
    __test_and_clear_bit((nr) ^ BITOP_LE_SWIZZLE, (addr))

#define ext2_clear_bit(nr, addr) \
    __test_and_clear_le_bit((nr), (unsigned long*)addr)

static inline int __test_and_clear_bit(int nr, volatile unsigned long *addr)
{
    unsigned long test = ((nr) % BITS_PER_LONG);
    unsigned long mask = BITOP_MASK(nr);
    unsigned long *p = ((unsigned long *)addr) + BITOP_WORD(nr);
    unsigned long old = *p;

    *p = old & ~mask;
    return (old & mask) != 0;
}
#define ext2_test_bit(nr, addr)      test_le_bit((nr),(unsigned long*)addr)
static __inline__ int test_le_bit(unsigned long nr,
                  __const__ unsigned long *addr)
{
    __const__ unsigned char    *tmp = (__const__ unsigned char *) addr;
    return (tmp[nr >> 3] >> (nr & 7)) & 1;
}

static __inline__ int test_and_clear_bit(unsigned long nr,
                                        volatile unsigned long *addr)
{
       unsigned long old, t;
       unsigned long mask = BITOP_MASK(nr);
       unsigned long *p = ((unsigned long *)addr) + BITOP_WORD(nr);

       __asm__ __volatile__(       "1:"    "ldarx %0,0,%3              # test_and_clear_bit\n"
       "andc  %1,%0,%2 \n"
       "dcbt 0,%3 \n"
       "stdcx. %1,0,%3 \n"
       "bne-   1b"
       : "=&r" (old), "=&r" (t)
       : "r" (mask), "r" (p)
       : "cc", "memory");

       return (old & mask) != 0;
}

main(int argc, char *argv[])
{
    char p[] = { 0xff, 0xff, 0xff, 0xff};
    char p2[] = { 0xff, 0xff, 0xff, 0xff};
    char *a = p + 1;
    //char *a = p;
    printf("is aligned %d\n", !((unsigned long )a) % 8);
    if (!strcmp(argv[1], "asm")) {
        test_and_clear_bit((1) ^ BITOP_LE_SWIZZLE, (unsigned long *)a);
    } else {
        ext2_clear_bit(1, a);
    }
    if (!ext2_test_bit(1, a)){
        printf("Bit is cleared\n");
    }
}

Ext4 progress

2007-10-13T22:59:00.000-07:00

Quiet a lot of exciting work happening with ext4. 2.6.24 is likely to have the uninitialized block group patch that helps in speeding up the e2fsck considerably. If i can get Alex to review the multiblock allocator change i made there is a high chance that that will also get in. 2.6.25 should have the large file size support for ext4. If you are interested in all these changes the place to look at is the ext4 patch queue.

Qemu GDB issue work around

2007-10-13T22:57:00.000-07:00

http://qemu-forum.ipi.fi/viewtopic.php?p=3281

flowers in the garden

2007-09-18T07:36:00.001-07:00

Painting

2007-09-15T08:08:00.000-07:00

Bought this one from here

Onam celebration at Samruddhi

2007-09-09T23:41:00.000-07:00

Mysore trip

2007-08-27T23:42:00.001-07:00

Mysore trip

2007-08-27T22:57:00.000-07:00

Mysore trip

2007-08-27T22:51:00.000-07:00

The Onam Feast - Sadhya

2007-08-24T01:19:00.001-07:00

India's Top Technical schools

2007-08-02T10:54:00.000-07:00

http://dqindia.ciol.com/content/top_stories/2007/107062205.asp

btrfs

2007-07-24T21:53:00.000-07:00

photos taken using the new k550i

One Laptop Per Child

2007-06-23T23:53:00.000-07:00

Btrfs from Chris Mason

2007-06-14T03:36:00.000-07:00

http://lwn.net/Articles/237904/

2007-06-11T08:28:00.000-07:00

New house

2007-06-11T08:26:00.000-07:00

We moved to the new house a month back.

House work

Linus on git

2007-05-20T21:32:00.000-07:00

gitview with annotation support

2007-04-22T12:41:00.000-07:00

ext4migrate

2007-03-19T04:21:00.000-07:00

Today i sent the patches to the list. You can find the same here http://article.gmane.org/gmane.comp.file-systems.ext4/1283 http://article.gmane.org/gmane.comp.file-systems.ext4/1284 http://article.gmane.org/gmane.comp.file-systems.ext4/1285 Currently it does only offline migration. I am looking at online defrag as a mechanism for implementing online migration.

Record number of candidates for Debian Project Leader election

2007-02-25T07:03:00.000-08:00

Here

B+ tree in ext4

2007-02-21T23:53:00.000-08:00

You may want to start from extents.c. In case you are looking for materials on B+ tree take a look at here and here

Threadlets

2007-02-21T22:21:00.000-08:00

Here. Well interesting in that it reduce the programming complexity. Take a look at threadlet-test.c in the test cases. You may want to spend some time around threadlet_exec. The trick is in restore stack and restore eip. Buddha blessed me only after having a discussion with bharata.

Git 1.5.0 release

2007-02-13T20:20:00.000-08:00

After a really exciting development cycle git 1.5.0 is released with quiet a lot of goodies

aio and syslets

2007-02-13T09:29:00.000-08:00

syslet intro syslet doc

3 years ago

2007-02-10T08:13:00.000-08:00

From Kuderamukh

Laptop RAM

2007-02-10T08:10:00.000-08:00

kvaneesh@skywalker:~$cat /proc/meminfo | head -n 1 MemTotal: 2074832 kB kvaneesh@skywalker:~$

vim ftplugin patch

2007-02-10T06:29:00.000-08:00

With the latest git the below changes are needed to get the ftplugin work

--- /tmp/git.vim 2007-02-10 19:58:12.000000000 +0530
+++ /home/kvaneesh/.vim/ftplugin/git.vim 2007-02-10 20:08:41.000000000 +0530
@@ -16,8 +16,8 @@
    " drop everything until '#  (will commit)' and the next empty line
    while i <= line('$')
        let line = getline(i)
-        if line =~ '^#\s*(will commit)$'
-            let i = i + 2
+        if line =~ '^#\s*Changes to be committed:$'
+            let i = i + 3
            break
        endif

google ajax search API

2007-02-05T10:12:00.000-08:00

Added google AJAX search to the blog page. I have added home page based custom search and blog search labels to the search result. That means one can use the top level blog page to search both my home page and blog. I am pretty impressed by the easiness with which one can develop these web apps. BTW i have "Head Rush AjAX" book lying in my book shelf. I am almost finished with the "Head First HTML". Right now i guess there is some bugs with the google's blog search APIs or may be it is the way i coded. It doesn't list results for all the keywords present in my blog page.

Nice LKML discussions

2007-02-04T00:40:00.000-08:00

CPU hotplug using freezer AIO using fibril Use kthreads instead of fibril

Migrating from livejournal to blogger

2007-02-01T06:47:00.000-08:00

Migrated all my journal entries from livejournal to blogger. The main reason for migrating was the ability to add multiple widgets to the blog page. So this would be my last livejournal entry. The new blog address is http://aneesh-kumar.blogspot.com/ I found this link useful in moving from livejournal to blogger .

Last day at HP

2007-01-21T22:50:00.000-08:00

Last Friday (19th Jan 2007) was my last day at HP. After 6 years 11 months and 19 days stay at HP, I felt a little bit awkward to say good bye. During this period I learnt quiet a lot. I was lucky to have a lot of nice friends. I also met my life partner there. A big thanks to HP for giving me such a wonderful time. I am moving to IBM. I will be joining the Linux technology center.

patches updated to 2.6.20-rc1

2006-12-30T23:30:00.000-08:00

I have updated LKCF patches to 2.6.20-rc1 kernel. gitweb link

Redhat Developer Day

2006-10-18T20:34:00.000-07:00

I was at Redhat Developer Day yesterday. Quiet a good event. Learnt about 108 and OLPC. I was there as the HP speaker. Being a developer day we decided to present developer oriented topics and this resulted in me presenting about Linux kernel Cluster Framework. I guess the presentation went fine.

LKCF selected for FOSS.IN 2006

2006-10-15T21:14:00.000-07:00

I haven't got any mail notification yet. But i can see it on the website here.

FOSS.IN/2006

2006-10-05T04:39:00.000-07:00

Submitted a talk on Linux kernel cluster framework (LKCF. Hopefully it will get selected. This is the first time i am submitting a talk for FOSS.IN.

Linux kernel Cluster Framework 0.2

2006-09-15T22:19:00.000-07:00

This is the 0.2 release of Linux kernel Cluster Framework.

What is LKCF:
-----------
LKCF's aim is to provide a transport independent cluster communication framework within the kernel. This enables the developers to write kernel based cluster services without being worried about communication transport. It also support RPC style programming. That means to write kernel service one need to write the service definition file (<*.svc>) and the implementation API. LKCF framework will generate all the registration routines and the marshaling code. Also it takes care of forwarding the SIGNALs across different nodes.The particular service can be called from any node specifying the node at which this particular service need to be executed. All the underlying management interface is taken care by the LKCF.

What transport are supported as of today:
---------------------------------------
IPV4

Infiniband verbs/RDMA.

Project Documentation:
---------------------
http://ci-linux.sourceforge.net/

Project git repository:
---------------------------
http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary

The patches can be found at http://git.openssi.org/~kvaneesh/ics_patches/lkcf-0.2/

and include

0001-Internode-communication-subsystem-for-Linux.txt
0002-ICS-over-Infiniband-verbs-work-in-progress.txt
0003-Token-facility-needed-for-cluster-based-synchronization.txt
0004-A-simple-test-case-for-ICS.txt

The patches are on top of git SHA1 ef7d1b244fa6c94fb76d5f787b8629df64ea4046 of the linus tree.

LKCF Infiniband support

2006-09-07T20:59:00.000-07:00

I have merged the work stan smith was doing with Infiniband and OpenSSI. I have pushed the changes here with changes to make it work with LKCF. I also tested the changes with a two node cluster with infiniband interconnect and it seems to work fine.

Linux Kernel Cluster Framework

2006-09-07T20:56:00.000-07:00

This is nothing but the CI rework i have been doing. The code has changed in such a way that it is not possible to build CI service module on the code base that i have without modification. So i decided to rename the work to LKCF.

Ubuntu's Upstart

2006-08-26T23:51:00.000-07:00

I really liked this. "upstart is a replacement for the init daemon, the process spawned by the kernel that is responsible for starting, supervising and stopping all other processes on the system." "upstart is an event-based init daemon; events generated by the system cause jobs to be started and running jobs to be stopped. Events can include things such as: * the system has started, * the root filesystem is now writable, * a block device has been added to the system, * a filesystem has been mounted, * at a certain time or repeated time period, * another job has begun running or has finished, * a file on the disk has been modified, * there are files in a queue directory, * a network device has been detected, * the default route has been added or removed." http://www.netsplit.com/blog/work/canonical/upstart.html

CI 0.1 released!!

2006-08-26T10:41:00.000-07:00

Today I released the 0.1 version of ICS. You can find the details here. Now from here there is quiet a lot of things that we can do such as: a) Add another transport. sctp, tipc, infiniband verbs etc. c) Get other cluster projects to start using it. From OpenSSI code i am trying to pull CFS code and other cluster wide SYSV IPC code. Once we get this done we will have clusterwide shared memory.

gitweb enhancements

2006-08-22T04:58:00.000-07:00

I have added support for urls and snapshots to gitweb. Both these changes are right now in next branch of git repository. The url support helps in finding out the the url that need to be used when you do a git-clone. The snapshot support helps you in getting a tar.gz image of the repository files at any commit point. This should help if you want to download the code without looking at the history. Along with that did some nice cleanups making blame and snapshot a feature which can be enabled/disabled via GITWEB_CONFIG and also project specific config. This should give you the details. http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary

Another milestone with CI

2006-08-21T08:47:00.000-07:00

Now that i have CI done i am looking at progressing with different components of OpenSSI cluster as below a) Get the token based synchronization working b) Get CFS working c) Get Clusterwide name server working d) Get message queue and semaphore working e) Get clusterwide shared memory working. I have done the first cut of cleanup with respect to token code and did a git push. You can find them here http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary. Follow the 2.6.18-rc1-ci-to-linus branch. BTW it is quiet a lot of work to get this done in free time. So in case you would like to help me let me know.

Cluster configuration via configfs

2006-06-27T21:18:00.000-07:00

I submitted all the changes required to build a CI cluster via configfs. This is really important in the sense it gets us on fast track in having a proper cluster manager in user space. I am right now looking at CMAN. With the current code base, one can configure the cluster with the below steps ifconfig eth0 192.168.2.15 netmask 255.255.255.0 mount -t configfs none /config mkdir /config/ics/node2 echo -n "192.168.2.15" > /config/ics/node2/icsinfo echo -n "2" > /config/ics/node2/nodenum ln -s /config/ics/node2 /config/ics/this_node ./cluster_config --preroot ./cluster_config --postroot echo -n "NODE_UP" > /config/ics/node2/status echo -n "NODE_READY" > /config/ics/node2/status To inform regarding other nodes status mkdir /config/ics/node3 echo -n "192.168.2.16" > /config/ics/node3/icsinfo echo -n "3" > /config/ics/node3/nodenum echo -n "NODE_CONFIG" > /config/ics/node3/status echo -n "NODE_UP" > /config/ics/node3/status echo -n "NODE_READY" > /config/ics/node3/status

Reworked Cluster Infrastructure code

2006-06-11T03:29:00.000-07:00

I have been working on CI code preparing it for submitting upstream. You can check the work here. Today i was able to run a simple test case of invoking a service on the second node. Right now the membership is configured from the user space via system call. I need to convert it to use configfs and CMAN.

git.openssi.org

2006-05-16T03:05:00.000-07:00

We now have git.openssi.org here

CI - A generic framework for internode cluster communication in Linux

2006-04-30T22:58:00.000-07:00

It is quiet a long time I updated my blog. That is mainly because I was busy cleaning up CI. Cluster infrastructure is an opensource project from HP. It implements a transport independent method of communicating across different nodes in a cluster. It also makes writing a cluster service extremly easy. Take a look at the documentation here. The reason for working on CI code is to get it merged with the vanilla Linux kernel (Linus kernel tree). Linux currently lacks a framework for writing cluster kernel services. Take a look at Oracle's OCFS or Redhat GFS. Both use their own mechanism for communicating to other nodes in a cluster. OCFs use TCP and GFS use sctp. I wanted to continue the process management work ( cluster proc ) done as a part of OpenSSI project and was looking at different ways of writing cluster services. That made me to look at CI code. The idea right now is to remove the kernel based cluster membership code and use Redhat CMAN. You should be able to find the status at here. I haven't pushed the code base yet. I have removed the CLMS code base and did quiet a lot of cleanups. Once i get the code compiled I will push it to radian. The second stage will be CMAN integration. Once i get CMAN integrated i would like to add one more lowlevel transport for CI. As a part of OpenSSI project Stan Smith had CI over Infiniband code base. I guess we didn't got it submitted to CVS. I will also try to get that integrated as well. I am also looking at having CI over TIPC . I know it is quiet a lot of work. So if anybody is interested let me know.

True!!

2006-03-09T00:19:00.000-08:00

After reading this "Concentration It's maybe a little fashionable to use terms like "information overload" and that all our technology is making us less productive. I'm starting to buy into this theory a little more though. I'm having a lot of trouble keeping myself focused on any one task for an extended period of time. I constantly jump between task A to task B to check email C, making a little progress here and there, but I think I'm hurting myself with all these context switches. I can't seem to stop myself anytime I have to wait for something (even very briefly), my focus switches to something else. The most problems are at work. Now I'm fairly sure the environment has a lot to do with it. It's an open concept type, with a few cubicle walls here and there. People are constantly interrupting me with questions and random conversations can break out frequently. For the most part I really like these aspects, but it can be very hard to concentrate on a task. Home isn't much better though. I have trouble just concentrating on one task, I almost always have a movie or TV going at the same time, and I can't seem to turn it off." more.. I found myself saying "same here" ( :( )

Git repository for OpenSSI

2006-03-06T03:57:00.000-08:00

Last few weeks i was trying to move the OpenSSI code from CVS to git. With OpenSSI we have a complicated tagging which even made viewcvs stop working. So my attempt to convert OpenSSI CVS to git using different git import export tools didn't work. So i ended up dropping all the code history and starting fresh with the 1.9.1 release. That means in the repository i have the base version of different packages and their respective openssi version as of 1.9.1 release. You can see the result of the work here
This may not be the final repository so if you are seriously working on OpenSSI code you should not base your development against these repositories. But I would very much like to get feedback regarding what others think of the current layout.

switching to bogofilter from spamassasin

2006-03-02T00:36:00.000-08:00

Using network manager for managing network with my laptop also means a mostly empty /etc/network/interfaces for me. This also resulted in spamd not starting properly. This made me look for other alternatives. I also had a feeling that spamassasin is too heavy for my usage. I decided to try bogofilter. Till now it is all working fine.

gitview 0.7 released

2006-02-24T23:22:00.000-08:00

Download from here Attaching below the change log -------------------------------------- commit 1509bd9e69b916ca7f04a89fc03662e03e2ba312 Author: Aneesh Kumar K.V Date: Fri Feb 24 21:57:51 2006 +0530 gitview: Fix the graph display . This fix all the known issue with the graph display The bug need to be explained graphically | a This line need not be there ---->| \ b | | / c c is parent of a and all a,b and c are placed on the same line and b is child of c With my last checkin I added a seperate line to indicate that a is connected to c. But then we had the line connecting a and b which should not be ther. This changes fixes the same bug Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit 9e4f522da7da8bf5f9018927c82b12e6b02b9058 Author: Aneesh Kumar K.V Date: Fri Feb 24 21:49:54 2006 +0530 gitview: Code cleanup Rearrange the code little bit so that it is easier to read Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit 20d23f554d6cd40ffa0d41ccc9416bca867667e0 Author: Aneesh Kumar Date: Fri Feb 24 14:08:35 2006 +0530 gitview: Bump the rev Make the 0.7 release Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit 8b42f5ae545d494463e72430fd81a0c0c558c881 Author: Aneesh Kumar Date: Fri Feb 24 14:02:32 2006 +0530 gitview: Fix DeprecationWarning DeprecationWarning: integer argument expected, got float Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit 3fe5489a2550118e0eb3f90e348cb78afeeb61e0 Author: Aneesh Kumar K.V Date: Fri Feb 24 00:59:42 2006 +0530 gitview: Display the lines joining commit nodes clearly. Since i wanted to limit the graph box size i was resetting the window after an index of 5. This result in line joining commit nodes to pass over nodes which are not related. The changes fixes the same Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit d27d5b3c5b97ca30dfc5c448dc8cdae914131051 Author: Junio C Hamano Date: Wed Feb 22 03:47:20 2006 -0800 gitview: ls-remote invocation shellquote safety. This will allow you to point GIT_DIR at directories with funny names. Signed-off-by: Junio C Hamano commit d800795613a710fb18353af53730e75185861f41 Author: aneesh.kumar@gmail.com Date: Tue Feb 21 16:00:10 2006 +0530 gitview: Use monospace font to draw the branch and tag name This patch address the below: Use monospace font to draw branch and tag name set the font size to 13. Make the graph column resizable. This helps to accommodate large tag names Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit 5301eee92ceb1c349ea8090a4d8aa3aa70c4abed Author: aneesh.kumar@gmail.com Date: Tue Feb 21 16:00:04 2006 +0530 gitview: Read tag and branch information using git ls-remote This fix the below bug Junio C Hamano writes: > > It does not work in my repository, since you do not seem to > handle branch and tag names with slashes in them. All of my > topic branches live in directories with two-letter names > (e.g. ak/gitview). Signed-off-by: Aneesh Kumar K.V Signed-off-by: Junio C Hamano commit c4d133a2b8501f0aded93fea147f0950db076ae1 Author: Aneesh Kumar K.V Date: Sat Feb 18 13:54:50 2006 +0530 gitview: typofix Signed-off-by: Aneesh Kumar K.V commit 020e3c1ee6924edbcf61b6e401058fa06ddbd406 Author: Aneesh Kumar K.V Date: Fri Feb 17 18:18:36 2006 +0530 Add a README for gitview Signed-off-by: Junio C Hamano commit 8cb711c8a5f98c1556b320114d7cc65e498ee2ae Author: Aneesh Kumar Date: Fri Feb 17 02:10:31 2006 -0800 Add contrib/gitview from Aneesh. Signed-off-by: Junio C Hamano

gitview now part of git project

2006-02-17T04:09:00.000-08:00

You can find it here here

Add support for saving patches

2006-02-13T20:41:00.000-08:00

With this you can save the changes by following the parent or child link in the middle panel.

gitview 0.5

2006-02-10T22:16:00.000-08:00

Released a new version of gitview. The change log is attached below. gitview can be downloaded from sf.net/projects/gitview commit 1dffd51311b6495adddcc1ff8682a61ba9209ff1 Author: Aneesh Kumar K.V Date: Tue Feb 7 10:22:03 2006 +0530 Support for displaying branch names and tag names commit 769c32fb71e5c3a2540ac9acab556d563d3984ee Author: Aneesh Kumar K.V Date: Mon Feb 6 10:18:41 2006 +0530 use the new --always flag with git diff-tree

Add branch and tag name display support

2006-02-07T00:00:00.000-08:00

gitview now hosted at sourceforge

2006-02-05T04:56:00.000-08:00

I have created a project at sourceforge for making future releases of gitview. If you are interested take a look at gitview

gitview 0.3

2006-02-01T20:40:00.000-08:00

I added an option to display diff in the main window. ( --with-diff ). Take a look at the snapshot below

You can find more details here gitview-0.3

Announce gitview-0.1

2006-01-31T20:36:00.000-08:00

Today i released the 0.1 version of gitview. I have requested to get it added to the git repository. I have added a new children link in the middle panel this time. This makes it easy to go to different commits. In case you haven't noticied, using CTRL-F will get you a search box that allows you to search with the Message field of the table.

gitview

2006-01-29T19:50:00.000-08:00

I was able to get the graph support working with gitview. This took me more time that i expected mostly because i was not able to follow the graph rendering class that scott had with bzrk. Here is the result

gitview

2006-01-26T20:13:00.000-08:00

One thing i didn't like with the packaged git tools was gitk. The tcl/tk script doesn't have the nice look and feel of the rest of the linux desktop. Since i am not good at writing a new GUI application myself, i started looking at utiities that i can fix to make them work with git. I finally found this one bzrk. After half day of hacking this is what i have . I am yet to get the graph part working. It is quiet complex. But i am sure i should be able to get it working soon.

Discussion with Mark Shuttleworth

2006-01-19T02:49:00.000-08:00

Today I had the chance to meet Mark Shuttleworth of ubuntu. We had some nice discussion regarding ubuntu and launchpad. My main question to him was about how he plan to keep the company running if he intends to develop only a distro that is given/shipped free of cost. Well quiet a lot of things became clear after we had the discussion. a) Why not debian: Getting/Forcing debian to work the way ubuntu guys wanted would have created trouble with debian. b) He is not intending to make "lot" of profit. He is more interested in the freedom part of the project. They don't ship 3rd party proprietary application on ubuntu. But they do that regarding some drivers . That in a way is nice as long as there is no other alternatives to get those hardware work with the distro c) launchpad : A collaboration framework where in all developers across distro can work together. Well if all that i heard in the meeting are true then it is going to be a killer webapp/website. d) why launchpad is closed source: Here comes the catch. His idea is to make ubuntu self sustaining. And the money for that comes from the service that cannonical offer The service offered by cannonical have a lot of dependency on launchpad. That is their value addition. He don't intend to make launchpad as a replacement for bugzilla or debian bug tracking. What launchpad is good at is, getting information out bugzilla or debian bug tracking and presenting a single page where in developers can track their bugs( there is much more to that ). Well the catch is the link between these different bug tracking system will be populated either by developer or the user who file the bug. All in all i found it interesting. I also asked him to write webblogs regarding different functionalities of launchpad so that all these can be understood much easily.

Got Married

2006-01-06T02:33:00.000-08:00

I have not been updating my blog for quiet a long time. There are many reason a) I was moved to a new team and had quiet a lot of work. I had no time to look at opensource especially OpenSSI. If you are wondering whether the new work is Linux related, No. But it is unix. b) Got married. Yes, I got married to Roopali on 11-Dec-2005. I am back to work now and i am going through my mails. Hopefully you should find me updating the blog more frequently now onwards.

2005-07-20T03:23:00.000-07:00

OpenSSI 1.9.1 released!!

2005-06-02T22:53:00.000-07:00

This release is interesting because we have integrated Xen ( http://www.cl.cam.ac.uk/Research/SRG/netos/xen/ ) with OpenSSI. Follow the below link if you want to try OpenSSI clustering with Xen http://www.openssi.org/cgi-bin/view?page=docs2/1.9/debian/xen-howto.txt.

OpenSSI and Xen

2005-05-13T04:05:00.000-07:00

Today i finished the first cut porting of OpenSSI to Xen. That means our 1.9.1 release of OpenSSI is going to support Xen virtual Machines !!!. All the changes are submitted to CVS. If you are a brave heart you can build the packages from CVS. The thing i like about Xen is the speed. I have tried to build a Cluster using UML, QEMU etc and all these attempts was dropped later because of the complexity involved in getting the configuration done or because of slowness. Xen is really fast and right now i have a two node cluster running on a single hardware and it's quiet fast. You can find the kernel patches for Xen ( which i haven't figured out how to put in the CVS ) at http://openssi.org/contrib/xen/

OpenSSI 1.9 release

2005-04-27T22:38:00.000-07:00

OpenSSI 1.9 release happened yesterday. Lot of exciting changes. Most important thing is, this release is based on the Linux kernel 2.6. That means if you wanted to run a cluster and also would like to have the new features available with the kernel 2.6 then this is the release you should try. Another big changes is that we made Debian release first !!!. That means Debian is now main distro. From my pet project to the main distro we release!! wow!!!. Now if you are going to give it a try I would ask you to try with a fresh install. We have tried to make sure an upgrade from 1.2 release works. But there could be some errors. If you are upgrading, please make sure you follow the upgrade document. ha-lvs got lot of enhancement. To find more look at the release notes at http://sourceforge.net/project/shownotes.php?group_id=32541&release_id=323532

OpenSSI development news

2005-03-10T01:56:00.000-08:00

I now have a two node Debian OpenSSI cluster running with 2.6 kernel. We are a lot stable with the 2.6 kernel now. Clusterwide Device story is the only remaining thing that is to be done with the 2.6 kernel branch. I have made quiet a few improvements with ha-lvs code in the development branch. This include moving all the hal-vs related /proc files to seq file interface and also making ha-lvs load balancer a configurable value. I have also changed the cvip configuration file format to make is much simpler to configure ha-lvs. DRBD (http://www.drbd.org/ ) had seen quiet a few changes in both the stable and the development branch. Configuring DRDB with Debian OpenSSI is a lot simpler since i have integrated all needed changes into tools. If you are interested follow the below document. http://www.openssi.org/cgi-bin/view?page=docs2/1.2/debian/README.drbd using DRBD in a OpenSSI cluster helps in building a high available cluster which can even take care of root node failure. BTW I started working full time for OpenSSI . This means i am moving from a voluntary mode to sponsored mode.

OpenSSI 1.2.1 release

2005-03-10T01:46:00.000-08:00

Released OpenSSI 1.2.1 for Debian. Follow the below document to install the Cluster. http://www.openssi.org/cgi-bin/view?page=docs2/1.2/debian/INSTALL.html Changes with respect 1.20 release include 2005/03/02 lramirez * Enabled some loadleveling code to take utime into consideration * Fix pgrp deadlock by allowing the rmv during reaps to be queued lock was busy. * Set all nsc kernel threads to run at higher priority. 2005/02/24 jlbyrne A summary of changes I've made since 1.2.0: * Fixed token bug with nodes higher than 32 * Various changes to logging. * Fixed CFS to stack over ramfs. * Chnaged CFS to mark pages up-to-date when a file is being extended. * Fixed panics in mount caused by NULL mount options. * Fixed siginfo reported with SIGCHLD. * Allowed processes with files open in /proc to migrate. * Allowed hostnames to be used in cvip.conf instead of numeric IP addresses. * Modified portmap not to die if an alias has the same address as a real interface. Allows the ICS address to be used as the CVIP gateway. 2005/02/24 kvaneesh * Fix drbd commands to be able to read the configuration file on a non OpenSSI kernel * Fix the drbd linuxrc to take care of split brain. (Fix suggested by Roger ) 2005/02/23 bjbrew * Correct some errors and omissions in README.ltsp 2005/02/21 lramirez * Fix bug in vpop_wait() where -ECHILD was being returned when waiting for specific children in a pgrp. Checked in by Roopa 2005/02/18 jlbyrne * Hack for allowing poll/selects to before `cluster_config --prep' 2005/02/17 kvaneesh * Validate the MAC and IP address while adding new cluster nodes * Fix the creation of global files by creating proper links 2005/02/15 gopalak * Merge distro-specific versions of README.nfs-server, README.nfs-client, and README.ntp into a single set of general documents 2005/02/08 kvaneesh * Add new device creation rules to debian devfsd. 2005/02/01 kvaneesh * Fix initrd-tools to create the device nodes correctly

OpenSSI 1.2 for Debian

2005-01-03T06:30:00.000-08:00

I forgot to update the Blog with the release information. OpenSSI 1.2 for Debian was released on 22-Dec-2004. Well to install follow the documents at http://www.openssi.org/docs/debian/INSTALL.shtml

One week away from Office

2005-01-03T06:21:00.000-08:00

Had a nice vacation at my home town. Finished the book The Da Vinci Code. Good one. Reading the book made me think about previous discussions that i had with my friends about twists in the Indian history. Earlier I had come across mails that state different opinions about the history of "tourists places" in India. I always kept wondering does it really matter ?. Does it really matter to anyone who built tajmahal and what lies beneath the great building ? For me it is an architectural masterpiece and is a monument for a good cause. So why do one need to change that ? What is wrong with the current way of looking at things ? Ofcourse it is good to know truth, but then at the end does it really matters if truth comes with pain and chaos ? BTW while arguing about "true history" with my friends, they said many Indians don't understand the "true Indian history" and knowing the history always brings in confidence. I would say that's their problem. Not due to the twist in the history. If you want to figure out Indian contributions to science read books on Bhagavad- Gita (psychology ), vedic mathematics (maths), Ayurveda (medicine) etc which are actually part of documented indian history..