introduction

TensorFlow, un cadre d'apprentissage en profondeur, réduit le basculement entre Python et GPU / ordinateurs distribués et rationalise les calculs en externalisant divers processus via des tampons de protocole.

Protocol Buffers est une technologie qui prend en charge l'informatique distribuée de Google et est un mécanisme de sérialisation des structures de données de manière indépendante du langage et de la plate-forme. Actuellement, C ++, C #, GO, Java et Python sont pris en charge.

TensorFlow est un mécanisme qui construit un graphique avec un traitement en tant que nœud et le calcule en même temps. Exemple de graphique

Ci-dessous, j'aimerais voir comment le graphique TensorFlow est sérialisé au format Protocol Buffers.

Exemple de programme

Ajoutons avec TensorFlow. Add 1 est exécuté 3 fois avec la valeur initiale 0.

`add.py`


import tensorflow as tf

state = tf.Variable(0, name="counter")

one = tf.constant(1)
new_value = tf.add(state, one)
update = tf.assign(state, new_value)

init_op = tf.initialize_all_variables()

with tf.Session() as sess:
  sess.run(init_op)
  print(sess.run(state))
  for _ in range(3):
    sess.run(update)
    print(sess.run(state))

Graphique TensorBoard

** Afficher le graphique avec TensorBoard **

Protocol Buffers Ci-dessous, nous verrons ce qui se passe lorsqu'un programme Python utilisant TensorFlow est converti en nœud Protocol Buffers.

Définition variable

Python

state = tf.Variable(0, name="counter")

Tampons de protocole: Variables nommées compteur

name: "counter"
op: "Variable"
attr {
  key: "container"
  value {
    s: ""
  }
}
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "shape"
  value {
    shape {
    }
  }
}
attr {
  key: "shared_name"
  value {
    s: ""
  }
}

Tampons de protocole: définition de la valeur initiale 0 (constante)

name: "counter/initial_value"
op: "Const"
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "value"
  value {
    tensor {
      dtype: DT_INT32
      tensor_shape {
      }
      int_val: 0
    }
  }
}

Tampons de protocole: attribuez la valeur initiale 0 au compteur variable

name: "counter/Assign"
op: "Assign"
input: "counter"
input: "counter/initial_value"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}
attr {
  key: "use_locking"
  value {
    b: true
  }
}
attr {
  key: "validate_shape"
  value {
    b: true
  }
}

Tampons de protocole: lecture du compteur de variables

name: "counter/read"
op: "Identity"
input: "counter"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}

Définition de constante (valeur supplémentaire)

Python

one = tf.constant(1)

Tampons de protocole: définition de la constante 1

name: "Const"
op: "Const"
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "value"
  value {
    tensor {
      dtype: DT_INT32
      tensor_shape {
      }
      int_val: 1
    }
  }
}

Définition de l'opération d'addition

Python

one = tf.constant(1)

Tampons de protocole: définir les opérations d'ajout

name: "Add"
op: "Add"
input: "counter/read"
input: "Const"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}

Remplacez le résultat de l'addition par une valeur dans une variable

Python

update = tf.assign(state, new_value)

Tampons de protocole: remplacez les valeurs des résultats d'addition dans les variables

name: "Assign"
op: "Assign"
input: "counter"
input: "Add"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}
attr {
  key: "use_locking"
  value {
    b: true
  }
}
attr {
  key: "validate_shape"
  value {
    b: true
  }
}

en conclusion

Si vous comprenez que TensorFlow utilise des tampons de protocole derrière votre programme Python, vous comprendrez pourquoi le traitement TensorFlow est branché à cet emplacement.

TensorFlow fournit divers processus de base (API).

Graphique
Variable
Tensol
Mathématiques
Chaîne
Syntaxe de contrôle
Assertion
Image
Vidéo
Entrée / sortie
Réseau neuronal
Résumé de l'opération
Test unitaire

Il est également intéressant d'essayer d'utiliser TensorFlow comme un cadre de calcul GPU / informatique distribué plutôt que comme un cadre d'apprentissage en profondeur.

(référence)

Programme d'affichage graphique

graph = tf.get_default_graph()
summary_writer = tf.train.SummaryWriter('log_valiable', graph)
operations =  graph.get_operations()
for operation in operations:
    print("======================")
    print("=== name ===")
    print(operation.name)
    print("=== type ===")
    print(operation.type)
    print("=== inputs ===")
    for input in operation.inputs:
        print(input)
    print("=== control_inputs ===")
    for control_input in operation.control_inputs:
        print(control_input)
    print("=== outputs ===")
    for output in operation.outputs:
        print(output)
    print("=== node_def ===")
    print(operation.node_def)
    print("=== op_def ===")
    print(operation.op_def)
    print("=== traceback ===")
    print(operation.traceback)
    print("")

Sortie de la console

0
1
2
3
======================
=== name ===
counter/initial_value
=== type ===
Const
=== inputs ===
=== control_inputs ===
=== outputs ===
Tensor("counter/initial_value:0", shape=(), dtype=int32)
=== node_def ===
name: "counter/initial_value"
op: "Const"
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "value"
  value {
    tensor {
      dtype: DT_INT32
      tensor_shape {
      }
      int_val: 0
    }
  }
}

=== op_def ===
None
=== traceback ===
[('./valiable.py', 5, '<module>', 'state = tf.Variable(0, name="counter")'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 215, '__init__', 'dtype=dtype)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 293, '_init_from_args', 'initial_value, name="initial_value", dtype=dtype)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 657, 'convert_to_tensor', 'ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py', 180, '_constant_tensor_conversion_function', 'return constant(v, dtype=dtype, name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py', 167, 'constant', 'attrs={"value": tensor_value, "dtype": dtype_value}, name=name).outputs[0]'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
counter
=== type ===
Variable
=== inputs ===
=== control_inputs ===
=== outputs ===
Tensor("counter:0", shape=(), dtype=int32_ref)
=== node_def ===
name: "counter"
op: "Variable"
attr {
  key: "container"
  value {
    s: ""
  }
}
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "shape"
  value {
    shape {
    }
  }
}
attr {
  key: "shared_name"
  value {
    s: ""
  }
}

=== op_def ===
name: "Variable"
output_arg {
  name: "ref"
  type_attr: "dtype"
  is_ref: true
}
attr {
  name: "shape"
  type: "shape"
}
attr {
  name: "dtype"
  type: "type"
}
attr {
  name: "container"
  type: "string"
  default_value {
    s: ""
  }
}
attr {
  name: "shared_name"
  type: "string"
  default_value {
    s: ""
  }
}
is_stateful: true

=== traceback ===
[('./valiable.py', 5, '<module>', 'state = tf.Variable(0, name="counter")'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 215, '__init__', 'dtype=dtype)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 300, '_init_from_args', 'name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/state_ops.py', 146, 'variable_op', 'container=container, shared_name=shared_name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_state_ops.py', 490, '_variable', 'name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py', 749, 'apply_op', 'op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
counter/Assign
=== type ===
Assign
=== inputs ===
Tensor("counter:0", shape=(), dtype=int32_ref)
Tensor("counter/initial_value:0", shape=(), dtype=int32)
=== control_inputs ===
=== outputs ===
Tensor("counter/Assign:0", shape=(), dtype=int32_ref)
=== node_def ===
name: "counter/Assign"
op: "Assign"
input: "counter"
input: "counter/initial_value"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}
attr {
  key: "use_locking"
  value {
    b: true
  }
}
attr {
  key: "validate_shape"
  value {
    b: true
  }
}

=== op_def ===
name: "Assign"
input_arg {
  name: "ref"
  type_attr: "T"
  is_ref: true
}
input_arg {
  name: "value"
  type_attr: "T"
}
output_arg {
  name: "output_ref"
  type_attr: "T"
  is_ref: true
}
attr {
  name: "T"
  type: "type"
}
attr {
  name: "validate_shape"
  type: "bool"
  default_value {
    b: true
  }
}
attr {
  name: "use_locking"
  type: "bool"
  default_value {
    b: true
  }
}
allows_uninitialized_input: true

=== traceback ===
[('./valiable.py', 5, '<module>', 'state = tf.Variable(0, name="counter")'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 215, '__init__', 'dtype=dtype)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 317, '_init_from_args', 'validate_shape=validate_shape).op'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_state_ops.py', 45, 'assign', 'use_locking=use_locking, name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py', 749, 'apply_op', 'op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
counter/read
=== type ===
Identity
=== inputs ===
Tensor("counter:0", shape=(), dtype=int32_ref)
=== control_inputs ===
=== outputs ===
Tensor("counter/read:0", shape=(), dtype=int32)
=== node_def ===
name: "counter/read"
op: "Identity"
input: "counter"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}

=== op_def ===
name: "Identity"
input_arg {
  name: "input"
  type_attr: "T"
}
output_arg {
  name: "output"
  type_attr: "T"
}
attr {
  name: "T"
  type: "type"
}

=== traceback ===
[('./valiable.py', 5, '<module>', 'state = tf.Variable(0, name="counter")'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 215, '__init__', 'dtype=dtype)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 327, '_init_from_args', 'self._snapshot = array_ops.identity(self._variable, name="read")'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py', 1128, 'identity', 'result = _op_def_lib.apply_op("Identity", input=input, name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py', 749, 'apply_op', 'op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
Const
=== type ===
Const
=== inputs ===
=== control_inputs ===
=== outputs ===
Tensor("Const:0", shape=(), dtype=int32)
=== node_def ===
name: "Const"
op: "Const"
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "value"
  value {
    tensor {
      dtype: DT_INT32
      tensor_shape {
      }
      int_val: 1
    }
  }
}

=== op_def ===
None
=== traceback ===
[('./valiable.py', 7, '<module>', 'one = tf.constant(1)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py', 167, 'constant', 'attrs={"value": tensor_value, "dtype": dtype_value}, name=name).outputs[0]'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
Add
=== type ===
Add
=== inputs ===
Tensor("counter/read:0", shape=(), dtype=int32)
Tensor("Const:0", shape=(), dtype=int32)
=== control_inputs ===
=== outputs ===
Tensor("Add:0", shape=(), dtype=int32)
=== node_def ===
name: "Add"
op: "Add"
input: "counter/read"
input: "Const"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}

=== op_def ===
name: "Add"
input_arg {
  name: "x"
  type_attr: "T"
}
input_arg {
  name: "y"
  type_attr: "T"
}
output_arg {
  name: "z"
  type_attr: "T"
}
attr {
  name: "T"
  type: "type"
  allowed_values {
    list {
      type: DT_HALF
      type: DT_FLOAT
      type: DT_DOUBLE
      type: DT_UINT8
      type: DT_INT8
      type: DT_INT16
      type: DT_INT32
      type: DT_INT64
      type: DT_COMPLEX64
      type: DT_COMPLEX128
      type: DT_STRING
    }
  }
}

=== traceback ===
[('./valiable.py', 8, '<module>', 'new_value = tf.add(state, one)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_math_ops.py', 71, 'add', 'result = _op_def_lib.apply_op("Add", x=x, y=y, name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py', 749, 'apply_op', 'op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
Assign
=== type ===
Assign
=== inputs ===
Tensor("counter:0", shape=(), dtype=int32_ref)
Tensor("Add:0", shape=(), dtype=int32)
=== control_inputs ===
=== outputs ===
Tensor("Assign:0", shape=(), dtype=int32_ref)
=== node_def ===
name: "Assign"
op: "Assign"
input: "counter"
input: "Add"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}
attr {
  key: "use_locking"
  value {
    b: true
  }
}
attr {
  key: "validate_shape"
  value {
    b: true
  }
}

=== op_def ===
name: "Assign"
input_arg {
  name: "ref"
  type_attr: "T"
  is_ref: true
}
input_arg {
  name: "value"
  type_attr: "T"
}
output_arg {
  name: "output_ref"
  type_attr: "T"
  is_ref: true
}
attr {
  name: "T"
  type: "type"
}
attr {
  name: "validate_shape"
  type: "bool"
  default_value {
    b: true
  }
}
attr {
  name: "use_locking"
  type: "bool"
  default_value {
    b: true
  }
}
allows_uninitialized_input: true

=== traceback ===
[('./valiable.py', 9, '<module>', 'update = tf.assign(state, new_value)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_state_ops.py', 45, 'assign', 'use_locking=use_locking, name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py', 749, 'apply_op', 'op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

======================
=== name ===
init
=== type ===
NoOp
=== inputs ===
=== control_inputs ===
name: "counter/Assign"
op: "Assign"
input: "counter"
input: "counter/initial_value"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@counter"
    }
  }
}
attr {
  key: "use_locking"
  value {
    b: true
  }
}
attr {
  key: "validate_shape"
  value {
    b: true
  }
}

=== outputs ===
=== node_def ===
name: "init"
op: "NoOp"
input: "^counter/Assign"

=== op_def ===
name: "NoOp"

=== traceback ===
[('./valiable.py', 11, '<module>', 'init_op = tf.initialize_all_variables()'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 1063, 'initialize_all_variables', 'return initialize_variables(all_variables())'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py', 1051, 'initialize_variables', 'return control_flow_ops.group(*[v.initializer for v in var_list], name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/control_flow_ops.py', 2645, 'group', 'return _GroupControlDeps(dev, deps, name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/control_flow_ops.py', 2603, '_GroupControlDeps', 'return no_op(name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_control_flow_ops.py', 184, 'no_op', 'result = _op_def_lib.apply_op("NoOp", name=name)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py', 756, 'apply_op', 'op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 2380, 'create_op', 'original_op=self._default_original_op, op_def=op_def)'), ('/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py', 1298, '__init__', 'self._traceback = _extract_stack()')]

Benchmark des tampons de protocole

J'ai en fait vérifié si les tampons de protocole sont vraiment lents @ 2016/8/24

[TensorFlow] Python ⇔ Tampons de protocole ⇔ GPU / Calcul distribué