This module provide an algorithm to evaluate a dataflow

exception core.algo.dataflow_evaluation.EvaluationException(vid, node, exception, exc_info)

Bases: exceptions.Exception

class core.algo.dataflow_evaluation.AbstractEvaluation(dataflow)

Bases: object

eval(*args)

todo

eval_vertex_code(vid)

Evaluate the vertex vid. Can raise an exception if evaluation failed.

get_parent_nodes(pid)

Return the list of parent node connected to pid The list contains tuples (port_pid, node_pid, actor) This list is sorted by the x value of the node

is_stopped(vid, actor)

Return True if evaluation must be stop at this vertex.

set_provenance(provenance)

class core.algo.dataflow_evaluation.BrutEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.AbstractEvaluation

Basic evaluation algorithm

eval(*args)

Evaluate the whole dataflow starting from leaves

eval_vertex(vid, *args)

Evaluate the vertex vid

is_stopped(vid, actor)

Return True if evaluation must be stop at this vertex

core.algo.dataflow_evaluation.DefaultEvaluation

alias of LambdaEvaluation

class core.algo.dataflow_evaluation.DiscreteTimeEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.AbstractEvaluation

Evaluation algorithm with generator / priority and selection

clear()

Clear evaluation variable

eval(vtx_id=None, step=False)

eval_vertex(vid)

Evaluate the vertex vid

is_stopped(vid, actor)

Return True if evaluation must be stop at this vertex

next_step()

Update the scheduler of one step.

class core.algo.dataflow_evaluation.GeneratorEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.AbstractEvaluation

Evaluation algorithm with generator / priority and selection

clear()

Clear evaluation variable

eval(vtx_id=None, step=False)

eval_vertex(vid)

Evaluate the vertex vid

is_stopped(vid, actor)

Return True if evaluation must be stop at this vertex

class core.algo.dataflow_evaluation.LambdaEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.PriorityEvaluation

Evaluation algorithm with support of lambda / priority and selection

eval(vtx_id=None, context=None, is_subdataflow=False, step=False)

Eval the dataflow from vtx_id with a particular context

Parameters:

• vtx_id -- vertex id to start the evaluation
• context -- list a value to assign to lambda variables

eval_vertex(vid, context, lambda_value, *args)

Evaluate the vertex vid

This function is called both by the user (eval a node and its parents) and by the SubDataFlow evaluation.

First the graph is traversed by the algorithm in a bottom-up way. The SubDataflow is stored in the inputs.

Parameters:context -- is a list a value to assign to lambdas

class core.algo.dataflow_evaluation.PrintProvenance(workflow)

Bases: core.algo.dataflow_evaluation.Provenance

node_exec(vid, node, start_time, end_time, *args)

workflow_exec(*args)

class core.algo.dataflow_evaluation.PriorityEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.BrutEvaluation

Support priority between nodes and selective

eval(vtx_id=None, *args, **kwds)

todo

class core.algo.dataflow_evaluation.Provenance(workflow)

Bases: object

clear()

edges()

end_time()

node_exec(vid, node, start_time, end_time, *args)

start_time()

workflow_exec(*args)

write()

Write the provenance in db

class core.algo.dataflow_evaluation.SciFlowareEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.AbstractEvaluation

Distributed Evaluation algorithm with SciFloware backend

eval(vtx_id=None, **kwds)

eval_vertex(vid)

Evaluate the vertex vid

This evaluation is both a kind of compilation and real evaluation. Algorithm --------- For each vertex which is a SciFloware operator (e.g. map, reduce, ...),

• select the vertices connected to each input port

• if the name of the port is Dataflow:

  • get its name and send it as input to the operator

• else

  • normal evaluation

static is_operator(actor)

scifloware_actors()

Compute the scifloware actors.

Only those actors will be evaluated.

class core.algo.dataflow_evaluation.ToScriptEvaluation(dataflow)

Bases: core.algo.dataflow_evaluation.AbstractEvaluation

Basic transformation into script algorithm

eval(*args, **kwds)

Evaluate the whole dataflow starting from leaves

eval_vertex(vid, *args)

Evaluate the vertex vid

is_stopped(vid, actor)

Return True if evaluation must be stop at this vertex

core.algo.dataflow_evaluation.cmp_posx(x, y)

todo

core.algo.dataflow_evaluation.cmp_priority(x, y)

todo

core.algo.dataflow_evaluation.db_connexion()

Return a curso on the database.

If the database does not exists, create it.

core.algo.dataflow_evaluation.db_create(cursor)

core.algo.dataflow_evaluation.get_database_name()

core.algo.dataflow_evaluation.provenance(vid, node, start_time, end_time)